2024-04-25T14:37:41.389 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T14:37:41.389 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} --verbose -- /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/orig.config.yaml 2024-04-25T14:37:41.446 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T14:37:41.543 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7673510' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8982 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG1z9CW5EAaB6BDadkeQ2MgtgSQXOHIq7VhMO0e6RHkkXxWjhAeygrR5NWKHONpGKG0guLfmkTAVkQYxUlswH78= smithi137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCDgTMsZivjEeq1T7k4rVha/1I/6nTMwHxMKLtWGcGc2P9DlZId7M4vylD8taa9Tte76dWn35Ar84+rkMt6kP2U= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEBSpKTrMbiaLLsuseKIzrGbGKf2e9kdMnDaAyKsyE8Iw06v1WU2yAqDycvyR6ZuCnt7JLXURvdZmjdUfcbRubQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_13:50:46 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T14:37:41.543 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa; will attempt to use it 2024-04-25T14:37:41.544 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks 2024-04-25T14:37:41.544 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T14:37:41.546 INFO:teuthology.task.internal:Checking packages... 2024-04-25T14:37:41.569 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T14:37:41.569 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T14:37:41.569 INFO:teuthology.packaging:ref: None 2024-04-25T14:37:41.570 INFO:teuthology.packaging:tag: None 2024-04-25T14:37:41.570 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T14:37:41.570 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:37:41.570 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T14:37:41.810 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T14:37:41.812 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T14:37:41.825 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T14:37:41.825 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T14:37:41.871 INFO:teuthology.task.internal:Saving configuration 2024-04-25T14:37:41.894 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T14:37:41.905 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T14:37:41.931 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:32:38.968570', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG1z9CW5EAaB6BDadkeQ2MgtgSQXOHIq7VhMO0e6RHkkXxWjhAeygrR5NWKHONpGKG0guLfmkTAVkQYxUlswH78='} 2024-04-25T14:37:41.956 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:32:38.970031', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCDgTMsZivjEeq1T7k4rVha/1I/6nTMwHxMKLtWGcGc2P9DlZId7M4vylD8taa9Tte76dWn35Ar84+rkMt6kP2U='} 2024-04-25T14:37:41.977 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:32:38.971293', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEBSpKTrMbiaLLsuseKIzrGbGKf2e9kdMnDaAyKsyE8Iw06v1WU2yAqDycvyR6ZuCnt7JLXURvdZmjdUfcbRubQ='} 2024-04-25T14:37:41.978 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T14:37:41.989 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T14:37:41.989 INFO:teuthology.task.internal:roles: ubuntu@smithi137.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T14:37:41.989 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T14:37:41.989 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T14:37:42.116 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff14b0eb0d0>, signals=[15]) 2024-04-25T14:37:42.116 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T14:37:42.124 INFO:teuthology.task.internal:Opening connections... 2024-04-25T14:37:42.125 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T14:37:42.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:37:42.208 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi137.front.sepia.ceph.com 2024-04-25T14:37:42.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:37:42.284 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T14:37:42.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:37:42.363 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T14:37:42.370 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-04-25T14:37:42.387 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-04-25T14:37:42.388 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:37:42.444 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-04-25T14:37:42.445 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:37:42.445 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:37:42.445 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:37:42.445 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:37:42.445 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:37:42.445 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:37:42.445 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-04-25T14:37:42.466 DEBUG:teuthology.orchestra.run.smithi137:> uname -m 2024-04-25T14:37:42.483 INFO:teuthology.orchestra.run.smithi137.stdout:x86_64 2024-04-25T14:37:42.483 DEBUG:teuthology.orchestra.run.smithi137:> cat /etc/os-release 2024-04-25T14:37:42.539 INFO:teuthology.orchestra.run.smithi137.stdout:NAME="CentOS Stream" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION="9" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:ID="centos" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:VERSION_ID="9" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:ANSI_COLOR="0;31" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:37:42.540 INFO:teuthology.orchestra.run.smithi137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:37:42.540 INFO:teuthology.lock.ops:Updating smithi137.front.sepia.ceph.com on lock server 2024-04-25T14:37:42.566 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-25T14:37:42.583 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-25T14:37:42.583 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-25T14:37:42.640 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-25T14:37:42.640 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-25T14:37:42.640 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-25T14:37:42.640 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T14:37:42.641 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T14:37:42.641 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-25T14:37:42.657 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T14:37:42.667 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T14:37:42.688 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T14:37:42.689 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:37:42.691 DEBUG:teuthology.orchestra.run.smithi137:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:37:42.693 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-25T14:37:42.709 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T14:37:42.720 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T14:37:42.720 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:37:42.747 DEBUG:teuthology.orchestra.run.smithi137:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:37:42.749 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-25T14:37:42.865 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T14:37:42.941 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T14:37:42.960 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T14:37:42.960 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T14:37:42.960 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T14:37:42.961 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:37:42.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:37:42.977 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T14:37:43.146 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:37:43.146 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T14:37:43.147 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-25T14:37:43.163 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-437.el9.x86_64 2024-04-25T14:37:43.164 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:37:43.164 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T14:37:43.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum upgrade -y linux-firmware 2024-04-25T14:37:45.534 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T14:37:47.544 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T14:37:52.036 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-25T14:37:53.610 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 16 kB 00:00 2024-04-25T14:38:13.288 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-25T14:38:18.360 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T14:38:20.331 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T14:38:20.332 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-04-25T14:38:20.332 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T14:38:20.444 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T14:38:20.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T14:38:20.747 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T14:38:20.747 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T14:38:20.750 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi094.front.sepia.ceph.com to complete... 2024-04-25T14:38:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25 14:38:46 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T14:38:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:FINISHED --2024-04-25 14:38:46-- 2024-04-25T14:38:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:Total wall clock time: 26s 2024-04-25T14:38:46.755 INFO:teuthology.orchestra.run.smithi094.stderr:Downloaded: 1 files, 341M in 25s (13.4 MB/s) 2024-04-25T14:38:46.758 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T14:38:46.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T14:38:48.834 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying... ######################################## 2024-04-25T14:38:48.940 INFO:teuthology.orchestra.run.smithi094.stdout:Preparing... ######################################## 2024-04-25T14:38:49.042 INFO:teuthology.orchestra.run.smithi094.stdout:Updating / installing... 2024-04-25T14:38:57.023 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T14:39:34.970 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T14:39:34.971 DEBUG:teuthology.orchestra.run.smithi094:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T14:39:35.047 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:35.047 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:35.047 INFO:teuthology.orchestra.run.smithi094.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:35.047 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T14:39:35.047 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T14:39:35.047 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T14:39:35.048 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T14:39:35.049 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T14:39:35.050 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T14:39:35.051 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T14:39:35.052 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T14:39:35.053 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T14:39:35.054 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T14:39:35.055 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T14:39:35.056 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T14:39:35.057 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T14:39:35.058 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T14:39:35.059 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T14:39:35.060 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T14:39:35.061 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T14:39:35.062 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T14:39:35.063 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T14:39:35.064 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T14:39:35.065 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T14:39:35.066 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T14:39:35.067 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T14:39:35.068 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T14:39:35.069 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T14:39:35.070 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T14:39:35.071 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T14:39:35.072 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T14:39:35.073 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T14:39:35.074 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T14:39:35.075 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T14:39:35.076 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T14:39:35.077 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T14:39:35.078 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T14:39:35.079 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T14:39:35.080 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T14:39:35.081 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T14:39:35.082 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T14:39:35.083 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T14:39:35.084 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T14:39:35.085 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T14:39:35.086 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T14:39:35.087 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T14:39:35.088 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T14:39:35.089 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T14:39:35.090 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T14:39:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T14:39:35.092 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T14:39:35.093 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T14:39:35.094 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T14:39:35.095 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T14:39:35.096 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T14:39:35.097 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T14:39:35.098 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T14:39:35.099 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T14:39:35.100 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T14:39:35.101 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T14:39:35.102 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T14:39:35.103 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T14:39:35.104 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T14:39:35.105 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T14:39:35.106 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T14:39:35.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T14:39:35.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T14:39:35.107 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T14:39:35.110 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T14:39:35.111 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T14:39:35.112 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T14:39:35.113 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T14:39:35.114 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T14:39:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T14:39:35.116 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T14:39:35.117 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T14:39:35.118 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T14:39:35.119 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T14:39:35.120 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T14:39:35.121 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T14:39:35.122 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T14:39:35.123 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T14:39:35.124 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T14:39:35.125 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T14:39:35.126 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T14:39:35.127 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T14:39:35.128 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T14:39:35.129 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T14:39:35.130 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T14:39:35.131 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T14:39:35.132 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T14:39:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T14:39:35.134 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T14:39:35.135 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T14:39:35.136 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T14:39:35.137 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T14:39:35.138 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T14:39:35.139 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T14:39:35.140 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T14:39:35.141 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T14:39:35.142 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T14:39:35.143 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T14:39:35.144 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T14:39:35.145 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T14:39:35.146 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T14:39:35.147 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T14:39:35.148 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T14:39:35.149 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T14:39:35.150 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T14:39:35.151 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T14:39:35.152 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T14:39:35.153 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T14:39:35.154 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T14:39:35.155 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T14:39:35.156 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T14:39:35.157 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T14:39:35.158 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T14:39:35.159 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T14:39:35.160 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T14:39:35.161 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T14:39:35.162 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T14:39:35.163 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T14:39:35.164 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T14:39:35.165 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T14:39:35.166 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T14:39:35.167 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T14:39:35.168 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T14:39:35.169 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T14:39:35.170 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T14:39:35.171 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T14:39:35.172 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T14:39:35.173 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T14:39:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T14:39:35.175 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T14:39:35.176 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T14:39:35.177 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T14:39:35.178 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T14:39:35.179 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T14:39:35.180 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T14:39:35.181 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T14:39:35.182 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T14:39:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T14:39:35.184 INFO:teuthology.orchestra.run.smithi094.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T14:39:35.188 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:35.188 DEBUG:teuthology.orchestra.run.smithi094:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T14:39:35.211 INFO:teuthology.orchestra.run.smithi094.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T14:39:35.211 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout:for file in vmlinuz System.map config; do 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout: fi 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout:done 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-25T14:39:35.212 INFO:teuthology.orchestra.run.smithi094.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T14:39:35.213 INFO:teuthology.orchestra.run.smithi094.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:35.213 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-25T14:39:35.213 INFO:teuthology.orchestra.run.smithi094.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T14:39:35.213 INFO:teuthology.orchestra.run.smithi094.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T14:39:35.213 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:35.213 INFO:teuthology.orchestra.run.smithi094.stdout:fi 2024-04-25T14:39:35.214 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-04-25T14:39:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-04-25T14:39:35.295 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:39:35.296 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T14:39:35.296 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T14:39:35.296 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:35.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T14:39:35.920 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-04-25T14:39:37.351 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T14:39:37.385 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-04-25T14:39:37.387 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-04-25T14:39:37.408 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.4fvOAkuHWa 2024-04-25T14:39:37.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4fvOAkuHWa 2024-04-25T14:39:37.473 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.4fvOAkuHWa 2024-04-25T14:39:37.618 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.4fvOAkuHWa is 6KB 2024-04-25T14:39:37.667 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.4fvOAkuHWa 2024-04-25T14:39:37.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T14:39:37.743 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T14:39:37.743 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T14:39:37.743 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T14:39:37.743 INFO:teuthology.orchestra.run.smithi094.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T14:39:37.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T14:39:37.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-04-25T14:40:07.892 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T14:40:07.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T14:40:07.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:40:26.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-04-25T14:40:35.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T14:40:35.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:41:35.403 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T14:41:47.404 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T14:41:47.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:41:47.790 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-25T14:41:48.403 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T14:41:48.403 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T14:41:48.403 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-04-25T14:41:48.454 INFO:teuthology.orchestra.run.smithi094.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:48.454 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T14:41:48.454 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T14:41:48.454 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T14:41:49.455 DEBUG:teuthology.parallel:result is None 2024-04-25T14:41:49.455 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T14:41:49.463 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T14:41:49.463 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:41:49.470 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:41:49.473 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T14:41:49.521 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T14:41:49.527 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T14:41:49.588 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T14:41:49.588 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:41:49.596 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:41:49.598 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T14:41:49.898 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T14:41:49.904 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T14:41:49.905 DEBUG:teuthology.orchestra.run.smithi137:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:41:49.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:41:49.921 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T14:41:49.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:41:49.938 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:41:49.945 DEBUG:teuthology.orchestra.run.smithi137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:41:49.965 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T14:41:50.011 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:41:50.018 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:41:50.036 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:41:50.039 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:41:50.075 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:41:50.139 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T14:41:50.144 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T14:41:50.150 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T14:41:50.151 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:41:50.158 DEBUG:teuthology.orchestra.run.smithi137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:41:50.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T14:41:50.237 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T14:41:50.247 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T14:41:50.247 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:41:50.252 DEBUG:teuthology.orchestra.run.smithi137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:41:50.255 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T14:41:50.301 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:41:50.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:41:50.480 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:41:50.565 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:41:50.650 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:41:50.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:41:50.734 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:41:50.779 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:41:50.835 DEBUG:teuthology.orchestra.run.smithi137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:41:50.918 DEBUG:teuthology.orchestra.run.smithi137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:41:50.960 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:41:50.960 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:41:51.026 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:41:51.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T14:41:51.386 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:41:51.430 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T14:41:51.473 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:41:51.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T14:41:51.540 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-04-25T14:41:51.546 DEBUG:teuthology.orchestra.run.smithi137:> sudo service rsyslog restart 2024-04-25T14:41:51.549 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-25T14:41:51.594 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:41:51.620 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:41:51.660 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T14:41:52.051 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T14:41:52.059 INFO:teuthology.task.internal:Starting timer... 2024-04-25T14:41:52.059 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T14:41:52.099 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T14:41:52.131 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T14:41:52.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-04-25T14:41:52.239 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-04-25T14:41:52.243 DEBUG:teuthology.orchestra.run.smithi137:> sudo service auditd rotate 2024-04-25T14:41:52.304 INFO:teuthology.orchestra.run.smithi137.stdout:Rotating logs: 2024-04-25T14:41:52.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-25T14:41:52.380 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-25T14:41:52.381 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T14:41:52.382 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-04-25T14:41:52.434 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-04-25T14:41:52.434 DEBUG:teuthology.orchestra.run.smithi137:> /usr/sbin/getenforce 2024-04-25T14:41:52.462 INFO:teuthology.orchestra.run.smithi137.stdout:Permissive 2024-04-25T14:41:52.462 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-25T14:41:52.484 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-25T14:41:52.484 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi137.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-04-25T14:41:52.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:41:52.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:41:52.557 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:41:52.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:41:52.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T14:41:52.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T14:41:52.611 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T14:41:52.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:41:52.680 DEBUG:teuthology.orchestra.run.smithi137:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:41:52.706 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-25T14:41:52.737 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T14:41:52.747 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T14:41:52.768 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T14:41:52.769 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi137.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T14:46:52.331 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi137.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-25T14:46:52.332 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T14:46:52.333 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:46:52.438 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-04-25T14:46:52.599 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-04-25T14:46:52.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-25T14:46:52.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:46:52.673 DEBUG:teuthology.orchestra.run.smithi137:> true 2024-04-25T14:46:52.754 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi137.front.sepia.ceph.com' 2024-04-25T14:46:52.754 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T14:46:52.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T14:46:52.830 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T14:46:52.912 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T14:46:52.913 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T14:46:52.923 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T14:46:52.924 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:46:52.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:46:52.932 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:46:52.932 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:46:52.935 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T14:46:52.935 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T14:46:52.962 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:46:52.977 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:46:52.982 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:46:52.997 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:46:53.001 INFO:teuthology.orchestra.run.smithi137.stderr:sudo: ntpd: command not found 2024-04-25T14:46:53.012 INFO:teuthology.orchestra.run.smithi137.stdout:506 Cannot talk to daemon 2024-04-25T14:46:53.013 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T14:46:53.021 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-25T14:46:53.025 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:46:53.032 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-25T14:46:53.039 INFO:teuthology.orchestra.run.smithi137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:46:53.047 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:46:53.052 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T14:46:53.060 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:46:53.102 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:46:53.107 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:46:53.107 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-25T14:46:53.107 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.107 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.107 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.107 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.115 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:46:53.116 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-04-25T14:46:53.118 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:46:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T14:46:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.119 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.147 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-04-25T14:46:53.187 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T14:46:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T14:46:53.345 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T14:46:53.353 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T14:46:53.353 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T14:46:53.353 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.354 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.354 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.354 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T14:46:53.355 INFO:teuthology.run_tasks:Running task install... 2024-04-25T14:46:53.364 DEBUG:teuthology.task.install:project ceph 2024-04-25T14:46:53.364 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T14:46:53.364 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T14:46:53.364 INFO:teuthology.task.install:Using flavor: default 2024-04-25T14:46:53.370 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T14:46:53.370 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T14:46:53.371 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:46:53.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:46:53.373 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:46:53.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:46:53.374 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T14:46:53.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:46:53.519 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:46:53.519 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:46:53.523 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:46:53.523 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:46:53.602 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T14:46:53.602 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T14:46:53.635 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:46:53.635 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:46:53.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:46:53.661 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:46:53.661 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:46:53.695 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:46:53.696 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:46:53.696 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:46:53.710 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T14:46:53.710 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:46:53.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T14:46:53.722 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:46:53.723 DEBUG:teuthology.orchestra.run.smithi137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:46:53.725 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:46:53.788 DEBUG:teuthology.orchestra.run.smithi137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:46:53.799 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:46:53.800 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T14:46:53.801 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T14:46:53.862 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-25T14:46:53.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-25T14:46:53.928 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T14:46:54.037 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:46:54.041 DEBUG:teuthology.orchestra.run.smithi137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T14:46:54.087 INFO:teuthology.orchestra.run.smithi138.stdout:48 files removed 2024-04-25T14:46:54.106 INFO:teuthology.orchestra.run.smithi137.stdout:check_obsoletes = 1 2024-04-25T14:46:54.107 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum clean all 2024-04-25T14:46:54.114 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-04-25T14:46:54.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:46:54.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-04-25T14:46:54.333 INFO:teuthology.orchestra.run.smithi137.stdout:48 files removed 2024-04-25T14:46:54.360 DEBUG:teuthology.orchestra.run.smithi137:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:46:54.434 INFO:teuthology.orchestra.run.smithi094.stdout:48 files removed 2024-04-25T14:46:54.473 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T14:46:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 240 kB/s | 83 kB 00:00 2024-04-25T14:46:55.098 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-04-25T14:46:55.106 INFO:teuthology.orchestra.run.smithi137.stdout:ceph packages for x86_64 170 kB/s | 83 kB 00:00 2024-04-25T14:46:55.483 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 146 kB/s | 83 kB 00:00 2024-04-25T14:46:55.507 INFO:teuthology.orchestra.run.smithi137.stdout:ceph noarch packages 33 kB/s | 12 kB 00:00 2024-04-25T14:46:55.507 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-25T14:46:55.832 INFO:teuthology.orchestra.run.smithi137.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-25T14:46:56.108 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-25T14:46:56.493 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T14:46:56.584 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-25T14:46:56.643 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-04-25T14:46:57.538 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.0 MB 00:00 2024-04-25T14:46:58.528 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T14:46:58.586 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-25T14:46:59.681 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T14:47:02.587 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T14:47:02.862 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 6.2 MB 00:00 2024-04-25T14:47:04.291 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-25T14:47:04.518 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 16 kB 00:00 2024-04-25T14:47:04.609 INFO:teuthology.orchestra.run.smithi137.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-25T14:47:05.703 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T14:47:06.186 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T14:47:07.228 INFO:teuthology.orchestra.run.smithi137.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 21 MB 00:02 2024-04-25T14:47:07.668 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T14:47:10.779 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-25T14:47:12.219 INFO:teuthology.orchestra.run.smithi137.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-25T14:47:12.904 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:47:12.905 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:47:12.916 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:47:12.917 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:47:12.917 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:47:12.918 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:47:12.919 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:47:12.922 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:47:12.922 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:47:12.982 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:47:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:47:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:47:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:47:13.008 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:47:13.009 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:47:13.010 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:47:13.011 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:47:13.012 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:47:13.013 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:47:13.014 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:47:13.015 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:47:13.016 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 4 Packages 2024-04-25T14:47:13.017 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:47:13.018 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 222 M 2024-04-25T14:47:13.018 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T14:47:13.613 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-25T14:47:13.827 INFO:teuthology.orchestra.run.smithi138.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 33 kB/s | 6.6 kB 00:00 2024-04-25T14:47:14.205 INFO:teuthology.orchestra.run.smithi138.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.3 MB/s | 890 kB 00:00 2024-04-25T14:47:14.305 INFO:teuthology.orchestra.run.smithi138.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-25T14:47:14.345 INFO:teuthology.orchestra.run.smithi137.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:47:14.346 INFO:teuthology.orchestra.run.smithi137.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:47:14.357 INFO:teuthology.orchestra.run.smithi137.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:47:14.358 INFO:teuthology.orchestra.run.smithi137.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:47:14.359 INFO:teuthology.orchestra.run.smithi137.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:47:14.362 INFO:teuthology.orchestra.run.smithi137.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:47:14.362 INFO:teuthology.orchestra.run.smithi137.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:47:14.363 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:47:14.364 INFO:teuthology.orchestra.run.smithi137.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:47:14.425 INFO:teuthology.orchestra.run.smithi137.stdout:Dependencies resolved. 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout: Package Arch Version Repository Size 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout:Installing: 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:47:14.453 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:47:14.454 INFO:teuthology.orchestra.run.smithi137.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:47:14.455 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrading: 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout:Installing dependencies: 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:47:14.456 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:47:14.457 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:47:14.458 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:47:14.459 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:47:14.460 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:47:14.461 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:47:14.462 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:47:14.463 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:47:14.464 INFO:teuthology.orchestra.run.smithi137.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout:Installing weak dependencies: 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction Summary 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout:======================================================================================= 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout:Install 148 Packages 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout:Upgrade 4 Packages 2024-04-25T14:47:14.465 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:47:14.466 INFO:teuthology.orchestra.run.smithi137.stdout:Total download size: 222 M 2024-04-25T14:47:14.466 INFO:teuthology.orchestra.run.smithi137.stdout:Downloading Packages: 2024-04-25T14:47:15.122 INFO:teuthology.orchestra.run.smithi138.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.9 MB/s | 2.4 MB 00:00 2024-04-25T14:47:15.329 INFO:teuthology.orchestra.run.smithi137.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-25T14:47:15.547 INFO:teuthology.orchestra.run.smithi138.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.5 MB/s | 1.5 MB 00:00 2024-04-25T14:47:15.656 INFO:teuthology.orchestra.run.smithi138.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-25T14:47:15.962 INFO:teuthology.orchestra.run.smithi137.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 890 kB 00:00 2024-04-25T14:47:16.023 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:47:16.024 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T14:47:16.045 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T14:47:16.046 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T14:47:16.047 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T14:47:16.049 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T14:47:16.051 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T14:47:16.055 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T14:47:16.056 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T14:47:16.121 INFO:teuthology.orchestra.run.smithi137.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 916 kB/s | 145 kB 00:00 2024-04-25T14:47:16.125 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-04-25T14:47:16.139 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T14:47:16.139 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-04-25T14:47:16.139 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T14:47:16.140 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T14:47:16.141 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T14:47:16.142 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T14:47:16.143 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T14:47:16.144 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T14:47:16.145 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T14:47:16.146 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T14:47:16.147 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T14:47:16.148 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T14:47:16.149 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-04-25T14:47:16.150 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout:Install 148 Packages 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 4 Packages 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 222 M 2024-04-25T14:47:16.151 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-04-25T14:47:17.231 INFO:teuthology.orchestra.run.smithi138.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.8 MB/s | 4.7 MB 00:01 2024-04-25T14:47:17.282 INFO:teuthology.orchestra.run.smithi094.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-25T14:47:17.421 INFO:teuthology.orchestra.run.smithi137.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-25T14:47:17.671 INFO:teuthology.orchestra.run.smithi137.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-25T14:47:18.190 INFO:teuthology.orchestra.run.smithi094.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 981 kB/s | 890 kB 00:00 2024-04-25T14:47:18.432 INFO:teuthology.orchestra.run.smithi094.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 602 kB/s | 145 kB 00:00 2024-04-25T14:47:18.463 INFO:teuthology.orchestra.run.smithi137.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 1.4 MB/s | 1.5 MB 00:01 2024-04-25T14:47:19.305 INFO:teuthology.orchestra.run.smithi137.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.9 MB/s | 4.7 MB 00:01 2024-04-25T14:47:19.374 INFO:teuthology.orchestra.run.smithi094.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-25T14:47:19.941 INFO:teuthology.orchestra.run.smithi094.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.6 MB/s | 1.5 MB 00:00 2024-04-25T14:47:20.583 INFO:teuthology.orchestra.run.smithi094.stdout:(6/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.1 MB/s | 2.4 MB 00:02 2024-04-25T14:47:21.859 INFO:teuthology.orchestra.run.smithi094.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 4.7 MB 00:01 2024-04-25T14:47:22.014 INFO:teuthology.orchestra.run.smithi138.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 2.2 MB/s | 11 MB 00:04 2024-04-25T14:47:22.106 INFO:teuthology.orchestra.run.smithi138.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 275 kB/s | 25 kB 00:00 2024-04-25T14:47:22.538 INFO:teuthology.orchestra.run.smithi137.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 3.3 MB/s | 11 MB 00:03 2024-04-25T14:47:22.638 INFO:teuthology.orchestra.run.smithi137.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 252 kB/s | 25 kB 00:00 2024-04-25T14:47:23.675 INFO:teuthology.orchestra.run.smithi094.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.1 MB/s | 21 MB 00:06 2024-04-25T14:47:23.778 INFO:teuthology.orchestra.run.smithi094.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 249 kB/s | 25 kB 00:00 2024-04-25T14:47:25.711 INFO:teuthology.orchestra.run.smithi094.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 2.7 MB/s | 11 MB 00:03 2024-04-25T14:47:25.911 INFO:teuthology.orchestra.run.smithi094.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 225 kB 00:00 2024-04-25T14:47:26.013 INFO:teuthology.orchestra.run.smithi094.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 272 kB/s | 27 kB 00:00 2024-04-25T14:47:26.120 INFO:teuthology.orchestra.run.smithi094.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 294 kB/s | 32 kB 00:00 2024-04-25T14:47:26.139 INFO:teuthology.orchestra.run.smithi138.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 21 MB 00:12 2024-04-25T14:47:26.547 INFO:teuthology.orchestra.run.smithi138.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 551 kB/s | 225 kB 00:00 2024-04-25T14:47:26.587 INFO:teuthology.orchestra.run.smithi094.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.6 MB/s | 747 kB 00:00 2024-04-25T14:47:26.631 INFO:teuthology.orchestra.run.smithi138.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 325 kB/s | 27 kB 00:00 2024-04-25T14:47:26.704 INFO:teuthology.orchestra.run.smithi094.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 163 kB 00:00 2024-04-25T14:47:26.724 INFO:teuthology.orchestra.run.smithi138.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 344 kB/s | 32 kB 00:00 2024-04-25T14:47:26.813 INFO:teuthology.orchestra.run.smithi094.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.1 MB/s | 126 kB 00:00 2024-04-25T14:47:27.138 INFO:teuthology.orchestra.run.smithi094.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 1.5 MB/s | 500 kB 00:00 2024-04-25T14:47:27.416 INFO:teuthology.orchestra.run.smithi138.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.1 MB/s | 747 kB 00:00 2024-04-25T14:47:27.691 INFO:teuthology.orchestra.run.smithi138.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 594 kB/s | 163 kB 00:00 2024-04-25T14:47:28.025 INFO:teuthology.orchestra.run.smithi138.stdout:(16/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 17 MB 00:12 2024-04-25T14:47:28.050 INFO:teuthology.orchestra.run.smithi138.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 353 kB/s | 126 kB 00:00 2024-04-25T14:47:28.164 INFO:teuthology.orchestra.run.smithi137.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 21 MB 00:13 2024-04-25T14:47:28.308 INFO:teuthology.orchestra.run.smithi138.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 1.7 MB/s | 500 kB 00:00 2024-04-25T14:47:28.355 INFO:teuthology.orchestra.run.smithi137.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.1 MB/s | 225 kB 00:00 2024-04-25T14:47:28.401 INFO:teuthology.orchestra.run.smithi138.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 490 kB/s | 45 kB 00:00 2024-04-25T14:47:28.456 INFO:teuthology.orchestra.run.smithi137.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 270 kB/s | 27 kB 00:00 2024-04-25T14:47:28.493 INFO:teuthology.orchestra.run.smithi138.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T14:47:28.564 INFO:teuthology.orchestra.run.smithi137.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 293 kB/s | 32 kB 00:00 2024-04-25T14:47:28.659 INFO:teuthology.orchestra.run.smithi138.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 969 kB/s | 161 kB 00:00 2024-04-25T14:47:28.876 INFO:teuthology.orchestra.run.smithi138.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 320 kB 00:00 2024-04-25T14:47:29.098 INFO:teuthology.orchestra.run.smithi137.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.4 MB/s | 747 kB 00:00 2024-04-25T14:47:29.118 INFO:teuthology.orchestra.run.smithi138.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 298 kB 00:00 2024-04-25T14:47:29.210 INFO:teuthology.orchestra.run.smithi138.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 99 kB 00:00 2024-04-25T14:47:29.255 INFO:teuthology.orchestra.run.smithi094.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 5.4 MB 00:02 2024-04-25T14:47:29.256 INFO:teuthology.orchestra.run.smithi137.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.0 MB/s | 163 kB 00:00 2024-04-25T14:47:29.269 INFO:teuthology.orchestra.run.smithi138.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T14:47:29.339 INFO:teuthology.orchestra.run.smithi094.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 549 kB/s | 45 kB 00:00 2024-04-25T14:47:29.348 INFO:teuthology.orchestra.run.smithi137.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.4 MB/s | 126 kB 00:00 2024-04-25T14:47:29.439 INFO:teuthology.orchestra.run.smithi094.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.3 MB/s | 135 kB 00:00 2024-04-25T14:47:29.556 INFO:teuthology.orchestra.run.smithi094.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.4 MB/s | 161 kB 00:00 2024-04-25T14:47:29.701 INFO:teuthology.orchestra.run.smithi137.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 1.4 MB/s | 500 kB 00:00 2024-04-25T14:47:29.740 INFO:teuthology.orchestra.run.smithi094.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 320 kB 00:00 2024-04-25T14:47:29.965 INFO:teuthology.orchestra.run.smithi094.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 298 kB 00:00 2024-04-25T14:47:30.091 INFO:teuthology.orchestra.run.smithi094.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 801 kB/s | 99 kB 00:00 2024-04-25T14:47:30.208 INFO:teuthology.orchestra.run.smithi094.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 735 kB/s | 85 kB 00:00 2024-04-25T14:47:30.744 INFO:teuthology.orchestra.run.smithi138.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.1 MB/s | 3.1 MB 00:01 2024-04-25T14:47:30.919 INFO:teuthology.orchestra.run.smithi138.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 975 kB/s | 170 kB 00:00 2024-04-25T14:47:31.011 INFO:teuthology.orchestra.run.smithi138.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 374 kB/s | 34 kB 00:00 2024-04-25T14:47:31.187 INFO:teuthology.orchestra.run.smithi138.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 812 kB/s | 142 kB 00:00 2024-04-25T14:47:31.341 INFO:teuthology.orchestra.run.smithi094.stdout:(26/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 17 MB 00:10 2024-04-25T14:47:31.433 INFO:teuthology.orchestra.run.smithi094.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.5 MB/s | 3.1 MB 00:01 2024-04-25T14:47:31.492 INFO:teuthology.orchestra.run.smithi094.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.1 MB/s | 170 kB 00:00 2024-04-25T14:47:31.517 INFO:teuthology.orchestra.run.smithi094.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 412 kB/s | 34 kB 00:00 2024-04-25T14:47:31.532 INFO:teuthology.orchestra.run.smithi137.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.3 MB/s | 17 MB 00:13 2024-04-25T14:47:31.584 INFO:teuthology.orchestra.run.smithi094.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.5 MB/s | 142 kB 00:00 2024-04-25T14:47:31.591 INFO:teuthology.orchestra.run.smithi137.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 773 kB/s | 45 kB 00:00 2024-04-25T14:47:31.716 INFO:teuthology.orchestra.run.smithi137.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.1 MB/s | 135 kB 00:00 2024-04-25T14:47:31.825 INFO:teuthology.orchestra.run.smithi137.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-25T14:47:32.075 INFO:teuthology.orchestra.run.smithi137.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 320 kB 00:00 2024-04-25T14:47:32.128 INFO:teuthology.orchestra.run.smithi138.stdout:(30/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 1.3 MB/s | 5.4 MB 00:04 2024-04-25T14:47:32.367 INFO:teuthology.orchestra.run.smithi137.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.0 MB/s | 298 kB 00:00 2024-04-25T14:47:32.525 INFO:teuthology.orchestra.run.smithi137.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 628 kB/s | 99 kB 00:00 2024-04-25T14:47:32.570 INFO:teuthology.orchestra.run.smithi138.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.6 MB/s | 3.5 MB 00:01 2024-04-25T14:47:32.626 INFO:teuthology.orchestra.run.smithi137.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 854 kB/s | 85 kB 00:00 2024-04-25T14:47:32.710 INFO:teuthology.orchestra.run.smithi094.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.0 MB/s | 3.5 MB 00:01 2024-04-25T14:47:32.720 INFO:teuthology.orchestra.run.smithi138.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.6 MB/s | 249 kB 00:00 2024-04-25T14:47:32.813 INFO:teuthology.orchestra.run.smithi138.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 537 kB/s | 49 kB 00:00 2024-04-25T14:47:32.818 INFO:teuthology.orchestra.run.smithi094.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.3 MB/s | 249 kB 00:00 2024-04-25T14:47:32.888 INFO:teuthology.orchestra.run.smithi138.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-25T14:47:32.902 INFO:teuthology.orchestra.run.smithi094.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 596 kB/s | 49 kB 00:00 2024-04-25T14:47:33.002 INFO:teuthology.orchestra.run.smithi094.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 165 kB/s | 16 kB 00:00 2024-04-25T14:47:33.105 INFO:teuthology.orchestra.run.smithi138.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 273 kB 00:00 2024-04-25T14:47:33.169 INFO:teuthology.orchestra.run.smithi094.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.6 MB/s | 273 kB 00:00 2024-04-25T14:47:33.201 INFO:teuthology.orchestra.run.smithi137.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 5.4 MB 00:03 2024-04-25T14:47:33.385 INFO:teuthology.orchestra.run.smithi137.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 932 kB/s | 170 kB 00:00 2024-04-25T14:47:33.422 INFO:teuthology.orchestra.run.smithi138.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.4 MB/s | 762 kB 00:00 2024-04-25T14:47:33.453 INFO:teuthology.orchestra.run.smithi094.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.6 MB/s | 762 kB 00:00 2024-04-25T14:47:33.501 INFO:teuthology.orchestra.run.smithi137.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 293 kB/s | 34 kB 00:00 2024-04-25T14:47:33.505 INFO:teuthology.orchestra.run.smithi138.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 219 kB/s | 18 kB 00:00 2024-04-25T14:47:33.553 INFO:teuthology.orchestra.run.smithi094.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 184 kB/s | 18 kB 00:00 2024-04-25T14:47:33.606 INFO:teuthology.orchestra.run.smithi138.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 805 kB/s | 80 kB 00:00 2024-04-25T14:47:33.652 INFO:teuthology.orchestra.run.smithi137.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 948 kB/s | 142 kB 00:00 2024-04-25T14:47:33.704 INFO:teuthology.orchestra.run.smithi094.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 539 kB/s | 80 kB 00:00 2024-04-25T14:47:33.706 INFO:teuthology.orchestra.run.smithi138.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-04-25T14:47:33.740 INFO:teuthology.orchestra.run.smithi138.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 5.7 MB/s | 195 kB 00:00 2024-04-25T14:47:33.782 INFO:teuthology.orchestra.run.smithi138.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T14:47:33.787 INFO:teuthology.orchestra.run.smithi094.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.5 MB/s | 803 kB 00:00 2024-04-25T14:47:33.830 INFO:teuthology.orchestra.run.smithi094.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 195 kB 00:00 2024-04-25T14:47:33.840 INFO:teuthology.orchestra.run.smithi138.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-04-25T14:47:33.871 INFO:teuthology.orchestra.run.smithi094.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-25T14:47:33.874 INFO:teuthology.orchestra.run.smithi138.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T14:47:33.907 INFO:teuthology.orchestra.run.smithi138.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.9 MB/s | 135 kB 00:00 2024-04-25T14:47:33.941 INFO:teuthology.orchestra.run.smithi138.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-04-25T14:47:33.947 INFO:teuthology.orchestra.run.smithi094.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.2 MB 00:00 2024-04-25T14:47:33.975 INFO:teuthology.orchestra.run.smithi138.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 6.3 MB/s | 215 kB 00:00 2024-04-25T14:47:33.989 INFO:teuthology.orchestra.run.smithi094.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T14:47:34.017 INFO:teuthology.orchestra.run.smithi138.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 16 MB/s | 689 kB 00:00 2024-04-25T14:47:34.031 INFO:teuthology.orchestra.run.smithi094.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T14:47:34.067 INFO:teuthology.orchestra.run.smithi138.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 18 MB/s | 943 kB 00:00 2024-04-25T14:47:34.073 INFO:teuthology.orchestra.run.smithi094.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-04-25T14:47:34.101 INFO:teuthology.orchestra.run.smithi138.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T14:47:34.115 INFO:teuthology.orchestra.run.smithi094.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-25T14:47:34.151 INFO:teuthology.orchestra.run.smithi138.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T14:47:34.165 INFO:teuthology.orchestra.run.smithi094.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T14:47:34.193 INFO:teuthology.orchestra.run.smithi138.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T14:47:34.224 INFO:teuthology.orchestra.run.smithi094.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T14:47:34.227 INFO:teuthology.orchestra.run.smithi138.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 981 kB/s | 33 kB 00:00 2024-04-25T14:47:34.266 INFO:teuthology.orchestra.run.smithi094.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T14:47:34.277 INFO:teuthology.orchestra.run.smithi137.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 1.9 MB/s | 3.1 MB 00:01 2024-04-25T14:47:34.300 INFO:teuthology.orchestra.run.smithi094.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 657 kB/s | 21 kB 00:00 2024-04-25T14:47:34.319 INFO:teuthology.orchestra.run.smithi138.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-25T14:47:34.359 INFO:teuthology.orchestra.run.smithi094.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-25T14:47:34.361 INFO:teuthology.orchestra.run.smithi138.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-25T14:47:34.394 INFO:teuthology.orchestra.run.smithi138.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-04-25T14:47:34.428 INFO:teuthology.orchestra.run.smithi138.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 160 kB 00:00 2024-04-25T14:47:34.462 INFO:teuthology.orchestra.run.smithi138.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T14:47:34.504 INFO:teuthology.orchestra.run.smithi138.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 662 kB 00:00 2024-04-25T14:47:34.546 INFO:teuthology.orchestra.run.smithi138.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-04-25T14:47:34.588 INFO:teuthology.orchestra.run.smithi138.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 613 kB/s | 25 kB 00:00 2024-04-25T14:47:34.621 INFO:teuthology.orchestra.run.smithi138.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-04-25T14:47:34.663 INFO:teuthology.orchestra.run.smithi138.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T14:47:34.697 INFO:teuthology.orchestra.run.smithi138.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 5.5 MB/s | 188 kB 00:00 2024-04-25T14:47:34.731 INFO:teuthology.orchestra.run.smithi138.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-04-25T14:47:34.871 INFO:teuthology.orchestra.run.smithi094.stdout:(52/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 4.4 MB/s | 49 MB 00:11 2024-04-25T14:47:34.897 INFO:teuthology.orchestra.run.smithi138.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-25T14:47:34.901 INFO:teuthology.orchestra.run.smithi094.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-04-25T14:47:35.056 INFO:teuthology.orchestra.run.smithi138.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-25T14:47:35.094 INFO:teuthology.orchestra.run.smithi137.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.5 MB/s | 3.5 MB 00:01 2024-04-25T14:47:35.123 INFO:teuthology.orchestra.run.smithi138.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-04-25T14:47:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.9 MB/s | 3.0 MB 00:00 2024-04-25T14:47:35.174 INFO:teuthology.orchestra.run.smithi138.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 18 MB/s | 911 kB 00:00 2024-04-25T14:47:35.202 INFO:teuthology.orchestra.run.smithi094.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 64 kB/s | 18 kB 00:00 2024-04-25T14:47:35.229 INFO:teuthology.orchestra.run.smithi137.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.8 MB/s | 249 kB 00:00 2024-04-25T14:47:35.229 INFO:teuthology.orchestra.run.smithi094.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 212 kB/s | 11 kB 00:00 2024-04-25T14:47:35.261 INFO:teuthology.orchestra.run.smithi094.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T14:47:35.274 INFO:teuthology.orchestra.run.smithi138.stdout:(69/152): ceph-mgr-diskprediction-local-19.0.0- 2.3 MB/s | 7.4 MB 00:03 2024-04-25T14:47:35.278 INFO:teuthology.orchestra.run.smithi094.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 942 kB/s | 45 kB 00:00 2024-04-25T14:47:35.294 INFO:teuthology.orchestra.run.smithi137.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 741 kB/s | 49 kB 00:00 2024-04-25T14:47:35.299 INFO:teuthology.orchestra.run.smithi138.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-25T14:47:35.320 INFO:teuthology.orchestra.run.smithi094.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T14:47:35.362 INFO:teuthology.orchestra.run.smithi094.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 7.4 MB/s | 582 kB 00:00 2024-04-25T14:47:35.386 INFO:teuthology.orchestra.run.smithi137.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 180 kB/s | 16 kB 00:00 2024-04-25T14:47:35.387 INFO:teuthology.orchestra.run.smithi094.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 383 kB/s | 25 kB 00:00 2024-04-25T14:47:35.413 INFO:teuthology.orchestra.run.smithi094.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-04-25T14:47:35.433 INFO:teuthology.orchestra.run.smithi138.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 37 MB/s | 5.8 MB 00:00 2024-04-25T14:47:35.446 INFO:teuthology.orchestra.run.smithi094.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.0 MB/s | 292 kB 00:00 2024-04-25T14:47:35.467 INFO:teuthology.orchestra.run.smithi138.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-04-25T14:47:35.480 INFO:teuthology.orchestra.run.smithi094.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 188 kB 00:00 2024-04-25T14:47:35.492 INFO:teuthology.orchestra.run.smithi138.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T14:47:35.505 INFO:teuthology.orchestra.run.smithi094.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 627 kB/s | 35 kB 00:00 2024-04-25T14:47:35.664 INFO:teuthology.orchestra.run.smithi094.stdout:(66/152): ceph-mgr-diskprediction-local-19.0.0- 1.8 MB/s | 7.4 MB 00:04 2024-04-25T14:47:35.720 INFO:teuthology.orchestra.run.smithi137.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-04-25T14:47:35.751 INFO:teuthology.orchestra.run.smithi138.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-04-25T14:47:35.776 INFO:teuthology.orchestra.run.smithi138.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 862 kB/s | 244 kB 00:00 2024-04-25T14:47:35.789 INFO:teuthology.orchestra.run.smithi094.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-25T14:47:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-25T14:47:35.843 INFO:teuthology.orchestra.run.smithi138.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 382 kB/s | 26 kB 00:00 2024-04-25T14:47:35.853 INFO:teuthology.orchestra.run.smithi137.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.6 MB/s | 762 kB 00:00 2024-04-25T14:47:35.877 INFO:teuthology.orchestra.run.smithi138.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-25T14:47:35.895 INFO:teuthology.orchestra.run.smithi137.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 440 kB/s | 18 kB 00:00 2024-04-25T14:47:35.902 INFO:teuthology.orchestra.run.smithi138.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T14:47:35.906 INFO:teuthology.orchestra.run.smithi094.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T14:47:35.920 INFO:teuthology.orchestra.run.smithi137.stdout:(38/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 511 kB/s | 273 kB 00:00 2024-04-25T14:47:35.927 INFO:teuthology.orchestra.run.smithi138.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-25T14:47:35.973 INFO:teuthology.orchestra.run.smithi094.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-04-25T14:47:36.007 INFO:teuthology.orchestra.run.smithi094.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-04-25T14:47:36.024 INFO:teuthology.orchestra.run.smithi094.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T14:47:36.053 INFO:teuthology.orchestra.run.smithi138.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-04-25T14:47:36.071 INFO:teuthology.orchestra.run.smithi137.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 80 kB 00:00 2024-04-25T14:47:36.086 INFO:teuthology.orchestra.run.smithi138.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T14:47:36.120 INFO:teuthology.orchestra.run.smithi138.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-25T14:47:36.129 INFO:teuthology.orchestra.run.smithi137.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-25T14:47:36.145 INFO:teuthology.orchestra.run.smithi138.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-25T14:47:36.155 INFO:teuthology.orchestra.run.smithi137.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-25T14:47:36.166 INFO:teuthology.orchestra.run.smithi094.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.5 MB/s | 1.0 MB 00:00 2024-04-25T14:47:36.170 INFO:teuthology.orchestra.run.smithi138.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-04-25T14:47:36.196 INFO:teuthology.orchestra.run.smithi138.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-25T14:47:36.205 INFO:teuthology.orchestra.run.smithi137.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T14:47:36.212 INFO:teuthology.orchestra.run.smithi138.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-25T14:47:36.255 INFO:teuthology.orchestra.run.smithi138.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T14:47:36.280 INFO:teuthology.orchestra.run.smithi138.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-04-25T14:47:36.297 INFO:teuthology.orchestra.run.smithi137.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 8.7 MB/s | 1.2 MB 00:00 2024-04-25T14:47:36.300 INFO:teuthology.orchestra.run.smithi094.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-04-25T14:47:36.305 INFO:teuthology.orchestra.run.smithi138.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T14:47:36.322 INFO:teuthology.orchestra.run.smithi137.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 916 kB/s | 106 kB 00:00 2024-04-25T14:47:36.341 INFO:teuthology.orchestra.run.smithi094.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 278 kB/s | 87 kB 00:00 2024-04-25T14:47:36.356 INFO:teuthology.orchestra.run.smithi137.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-25T14:47:36.389 INFO:teuthology.orchestra.run.smithi137.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T14:47:36.431 INFO:teuthology.orchestra.run.smithi137.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T14:47:36.473 INFO:teuthology.orchestra.run.smithi137.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 8.0 MB/s | 689 kB 00:00 2024-04-25T14:47:36.508 INFO:teuthology.orchestra.run.smithi094.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-25T14:47:36.524 INFO:teuthology.orchestra.run.smithi137.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-25T14:47:36.542 INFO:teuthology.orchestra.run.smithi094.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T14:47:36.549 INFO:teuthology.orchestra.run.smithi137.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-25T14:47:36.574 INFO:teuthology.orchestra.run.smithi137.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-25T14:47:36.575 INFO:teuthology.orchestra.run.smithi094.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 248 kB 00:00 2024-04-25T14:47:36.589 INFO:teuthology.orchestra.run.smithi138.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 147 kB/s | 42 kB 00:00 2024-04-25T14:47:36.599 INFO:teuthology.orchestra.run.smithi137.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T14:47:36.601 INFO:teuthology.orchestra.run.smithi094.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 293 kB/s | 26 kB 00:00 2024-04-25T14:47:36.625 INFO:teuthology.orchestra.run.smithi137.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 660 kB/s | 33 kB 00:00 2024-04-25T14:47:36.626 INFO:teuthology.orchestra.run.smithi094.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 391 kB/s | 19 kB 00:00 2024-04-25T14:47:36.651 INFO:teuthology.orchestra.run.smithi094.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-25T14:47:36.685 INFO:teuthology.orchestra.run.smithi094.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 425 kB/s | 35 kB 00:00 2024-04-25T14:47:36.700 INFO:teuthology.orchestra.run.smithi137.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T14:47:36.725 INFO:teuthology.orchestra.run.smithi137.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 186 kB/s | 18 kB 00:00 2024-04-25T14:47:36.744 INFO:teuthology.orchestra.run.smithi094.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T14:47:36.751 INFO:teuthology.orchestra.run.smithi137.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-25T14:47:36.772 INFO:teuthology.orchestra.run.smithi138.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-04-25T14:47:36.776 INFO:teuthology.orchestra.run.smithi137.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T14:47:36.798 INFO:teuthology.orchestra.run.smithi138.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 214 kB/s | 44 kB 00:00 2024-04-25T14:47:36.801 INFO:teuthology.orchestra.run.smithi137.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T14:47:36.823 INFO:teuthology.orchestra.run.smithi138.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T14:47:36.827 INFO:teuthology.orchestra.run.smithi137.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T14:47:36.849 INFO:teuthology.orchestra.run.smithi138.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-04-25T14:47:36.860 INFO:teuthology.orchestra.run.smithi137.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T14:47:36.874 INFO:teuthology.orchestra.run.smithi138.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T14:47:36.878 INFO:teuthology.orchestra.run.smithi094.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-25T14:47:36.886 INFO:teuthology.orchestra.run.smithi137.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 443 kB/s | 25 kB 00:00 2024-04-25T14:47:36.899 INFO:teuthology.orchestra.run.smithi138.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T14:47:36.911 INFO:teuthology.orchestra.run.smithi137.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T14:47:36.911 INFO:teuthology.orchestra.run.smithi094.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-04-25T14:47:36.924 INFO:teuthology.orchestra.run.smithi138.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T14:47:36.936 INFO:teuthology.orchestra.run.smithi137.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T14:47:36.936 INFO:teuthology.orchestra.run.smithi094.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 437 kB/s | 77 kB 00:00 2024-04-25T14:47:36.950 INFO:teuthology.orchestra.run.smithi138.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T14:47:36.961 INFO:teuthology.orchestra.run.smithi137.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T14:47:36.962 INFO:teuthology.orchestra.run.smithi094.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 526 kB/s | 42 kB 00:00 2024-04-25T14:47:36.975 INFO:teuthology.orchestra.run.smithi138.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T14:47:36.986 INFO:teuthology.orchestra.run.smithi137.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 709 kB/s | 35 kB 00:00 2024-04-25T14:47:37.000 INFO:teuthology.orchestra.run.smithi138.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-04-25T14:47:37.004 INFO:teuthology.orchestra.run.smithi094.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-04-25T14:47:37.026 INFO:teuthology.orchestra.run.smithi138.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-25T14:47:37.029 INFO:teuthology.orchestra.run.smithi094.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-25T14:47:37.062 INFO:teuthology.orchestra.run.smithi094.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 279 kB 00:00 2024-04-25T14:47:37.068 INFO:teuthology.orchestra.run.smithi138.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T14:47:37.080 INFO:teuthology.orchestra.run.smithi094.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 51 kB 00:00 2024-04-25T14:47:37.096 INFO:teuthology.orchestra.run.smithi094.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 848 kB/s | 54 kB 00:00 2024-04-25T14:47:37.104 INFO:teuthology.orchestra.run.smithi137.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-25T14:47:37.179 INFO:teuthology.orchestra.run.smithi137.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-04-25T14:47:37.212 INFO:teuthology.orchestra.run.smithi137.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T14:47:37.246 INFO:teuthology.orchestra.run.smithi137.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 14 MB/s | 911 kB 00:00 2024-04-25T14:47:37.271 INFO:teuthology.orchestra.run.smithi137.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T14:47:37.297 INFO:teuthology.orchestra.run.smithi094.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 226 kB/s | 42 kB 00:00 2024-04-25T14:47:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-04-25T14:47:37.314 INFO:teuthology.orchestra.run.smithi094.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 220 kB/s | 44 kB 00:00 2024-04-25T14:47:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T14:47:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-04-25T14:47:37.402 INFO:teuthology.orchestra.run.smithi138.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 518 kB/s | 26 kB 00:00 2024-04-25T14:47:37.430 INFO:teuthology.orchestra.run.smithi137.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 32 MB/s | 5.8 MB 00:00 2024-04-25T14:47:37.463 INFO:teuthology.orchestra.run.smithi137.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-25T14:47:37.477 INFO:teuthology.orchestra.run.smithi138.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 649 kB/s | 49 kB 00:00 2024-04-25T14:47:37.488 INFO:teuthology.orchestra.run.smithi137.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T14:47:37.536 INFO:teuthology.orchestra.run.smithi138.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T14:47:37.603 INFO:teuthology.orchestra.run.smithi138.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-25T14:47:37.623 INFO:teuthology.orchestra.run.smithi094.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-04-25T14:47:37.631 INFO:teuthology.orchestra.run.smithi137.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-04-25T14:47:37.648 INFO:teuthology.orchestra.run.smithi094.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 631 kB/s | 221 kB 00:00 2024-04-25T14:47:37.653 INFO:teuthology.orchestra.run.smithi138.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T14:47:37.656 INFO:teuthology.orchestra.run.smithi137.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-25T14:47:37.681 INFO:teuthology.orchestra.run.smithi137.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T14:47:37.687 INFO:teuthology.orchestra.run.smithi138.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-25T14:47:37.707 INFO:teuthology.orchestra.run.smithi137.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 507 kB/s | 26 kB 00:00 2024-04-25T14:47:37.729 INFO:teuthology.orchestra.run.smithi138.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T14:47:37.731 INFO:teuthology.orchestra.run.smithi094.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 89 kB/s | 36 kB 00:00 2024-04-25T14:47:37.732 INFO:teuthology.orchestra.run.smithi137.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-25T14:47:37.746 INFO:teuthology.orchestra.run.smithi138.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-04-25T14:47:37.757 INFO:teuthology.orchestra.run.smithi137.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T14:47:37.771 INFO:teuthology.orchestra.run.smithi138.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 769 kB/s | 32 kB 00:00 2024-04-25T14:47:37.781 INFO:teuthology.orchestra.run.smithi094.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 974 kB/s | 47 kB 00:00 2024-04-25T14:47:37.783 INFO:teuthology.orchestra.run.smithi137.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-04-25T14:47:37.788 INFO:teuthology.orchestra.run.smithi138.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 357 kB/s | 15 kB 00:00 2024-04-25T14:47:37.807 INFO:teuthology.orchestra.run.smithi094.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T14:47:37.830 INFO:teuthology.orchestra.run.smithi138.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-25T14:47:37.840 INFO:teuthology.orchestra.run.smithi094.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 542 kB/s | 58 kB 00:00 2024-04-25T14:47:37.855 INFO:teuthology.orchestra.run.smithi138.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-25T14:47:37.906 INFO:teuthology.orchestra.run.smithi138.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-25T14:47:37.915 INFO:teuthology.orchestra.run.smithi094.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T14:47:37.931 INFO:teuthology.orchestra.run.smithi138.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 470 kB/s | 35 kB 00:00 2024-04-25T14:47:37.940 INFO:teuthology.orchestra.run.smithi094.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 644 kB/s | 64 kB 00:00 2024-04-25T14:47:37.950 INFO:teuthology.orchestra.run.smithi137.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-04-25T14:47:37.957 INFO:teuthology.orchestra.run.smithi094.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-04-25T14:47:37.964 INFO:teuthology.orchestra.run.smithi138.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-25T14:47:37.975 INFO:teuthology.orchestra.run.smithi137.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-04-25T14:47:37.991 INFO:teuthology.orchestra.run.smithi094.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 695 kB/s | 22 kB 00:00 2024-04-25T14:47:38.000 INFO:teuthology.orchestra.run.smithi137.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T14:47:38.015 INFO:teuthology.orchestra.run.smithi138.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 24 MB/s | 2.0 MB 00:00 2024-04-25T14:47:38.026 INFO:teuthology.orchestra.run.smithi137.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T14:47:38.040 INFO:teuthology.orchestra.run.smithi138.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T14:47:38.051 INFO:teuthology.orchestra.run.smithi137.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-04-25T14:47:38.150 INFO:teuthology.orchestra.run.smithi094.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-04-25T14:47:38.159 INFO:teuthology.orchestra.run.smithi138.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 74 kB/s | 11 kB 00:00 2024-04-25T14:47:38.176 INFO:teuthology.orchestra.run.smithi137.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-04-25T14:47:38.199 INFO:teuthology.orchestra.run.smithi138.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 111 kB/s | 18 kB 00:00 2024-04-25T14:47:38.210 INFO:teuthology.orchestra.run.smithi137.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-04-25T14:47:38.224 INFO:teuthology.orchestra.run.smithi138.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 359 kB/s | 23 kB 00:00 2024-04-25T14:47:38.235 INFO:teuthology.orchestra.run.smithi137.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-25T14:47:38.249 INFO:teuthology.orchestra.run.smithi138.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-25T14:47:38.260 INFO:teuthology.orchestra.run.smithi137.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T14:47:38.275 INFO:teuthology.orchestra.run.smithi138.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-04-25T14:47:38.285 INFO:teuthology.orchestra.run.smithi137.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T14:47:38.325 INFO:teuthology.orchestra.run.smithi138.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-04-25T14:47:38.367 INFO:teuthology.orchestra.run.smithi138.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-25T14:47:38.393 INFO:teuthology.orchestra.run.smithi138.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-04-25T14:47:38.418 INFO:teuthology.orchestra.run.smithi138.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T14:47:38.451 INFO:teuthology.orchestra.run.smithi138.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-25T14:47:38.477 INFO:teuthology.orchestra.run.smithi138.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-25T14:47:38.510 INFO:teuthology.orchestra.run.smithi138.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-25T14:47:38.511 INFO:teuthology.orchestra.run.smithi137.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 185 kB/s | 42 kB 00:00 2024-04-25T14:47:38.517 INFO:teuthology.orchestra.run.smithi094.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 983 kB/s | 548 kB 00:00 2024-04-25T14:47:38.535 INFO:teuthology.orchestra.run.smithi138.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-04-25T14:47:38.543 INFO:teuthology.orchestra.run.smithi094.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 59 kB/s | 19 kB 00:00 2024-04-25T14:47:38.561 INFO:teuthology.orchestra.run.smithi138.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-04-25T14:47:38.576 INFO:teuthology.orchestra.run.smithi094.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 606 kB/s | 309 kB 00:00 2024-04-25T14:47:38.594 INFO:teuthology.orchestra.run.smithi138.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-25T14:47:38.620 INFO:teuthology.orchestra.run.smithi138.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T14:47:38.645 INFO:teuthology.orchestra.run.smithi138.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-04-25T14:47:38.670 INFO:teuthology.orchestra.run.smithi138.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T14:47:38.703 INFO:teuthology.orchestra.run.smithi137.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-04-25T14:47:38.704 INFO:teuthology.orchestra.run.smithi138.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-25T14:47:38.721 INFO:teuthology.orchestra.run.smithi138.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-25T14:47:38.728 INFO:teuthology.orchestra.run.smithi137.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 205 kB/s | 44 kB 00:00 2024-04-25T14:47:38.753 INFO:teuthology.orchestra.run.smithi137.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T14:47:38.763 INFO:teuthology.orchestra.run.smithi138.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-04-25T14:47:38.779 INFO:teuthology.orchestra.run.smithi137.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-04-25T14:47:38.788 INFO:teuthology.orchestra.run.smithi138.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-25T14:47:38.804 INFO:teuthology.orchestra.run.smithi137.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T14:47:38.822 INFO:teuthology.orchestra.run.smithi138.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-25T14:47:38.829 INFO:teuthology.orchestra.run.smithi137.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T14:47:38.855 INFO:teuthology.orchestra.run.smithi137.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T14:47:38.872 INFO:teuthology.orchestra.run.smithi138.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-04-25T14:47:38.880 INFO:teuthology.orchestra.run.smithi137.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T14:47:38.905 INFO:teuthology.orchestra.run.smithi137.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T14:47:38.930 INFO:teuthology.orchestra.run.smithi137.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-04-25T14:47:38.955 INFO:teuthology.orchestra.run.smithi137.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T14:47:38.989 INFO:teuthology.orchestra.run.smithi137.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T14:47:39.136 INFO:teuthology.orchestra.run.smithi094.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-04-25T14:47:39.161 INFO:teuthology.orchestra.run.smithi094.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 42 kB/s | 26 kB 00:00 2024-04-25T14:47:39.186 INFO:teuthology.orchestra.run.smithi094.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 80 kB/s | 49 kB 00:00 2024-04-25T14:47:39.211 INFO:teuthology.orchestra.run.smithi094.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-04-25T14:47:39.236 INFO:teuthology.orchestra.run.smithi094.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-25T14:47:39.306 INFO:teuthology.orchestra.run.smithi137.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-04-25T14:47:39.331 INFO:teuthology.orchestra.run.smithi137.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 904 kB/s | 309 kB 00:00 2024-04-25T14:47:39.359 INFO:teuthology.orchestra.run.smithi137.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 374 kB/s | 19 kB 00:00 2024-04-25T14:47:39.362 INFO:teuthology.orchestra.run.smithi094.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-25T14:47:39.404 INFO:teuthology.orchestra.run.smithi094.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-04-25T14:47:39.429 INFO:teuthology.orchestra.run.smithi094.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 231 kB/s | 43 kB 00:00 2024-04-25T14:47:39.434 INFO:teuthology.orchestra.run.smithi137.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 350 kB/s | 26 kB 00:00 2024-04-25T14:47:39.454 INFO:teuthology.orchestra.run.smithi094.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 364 kB/s | 32 kB 00:00 2024-04-25T14:47:39.479 INFO:teuthology.orchestra.run.smithi094.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 306 kB/s | 15 kB 00:00 2024-04-25T14:47:39.492 INFO:teuthology.orchestra.run.smithi137.stdout:(108/152): libarrow-9.0.0-10.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2024-04-25T14:47:39.505 INFO:teuthology.orchestra.run.smithi094.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-04-25T14:47:39.518 INFO:teuthology.orchestra.run.smithi137.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 585 kB/s | 49 kB 00:00 2024-04-25T14:47:39.538 INFO:teuthology.orchestra.run.smithi094.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.3 MB/s | 358 kB 00:00 2024-04-25T14:47:39.543 INFO:teuthology.orchestra.run.smithi137.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T14:47:39.572 INFO:teuthology.orchestra.run.smithi094.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.7 MB/s | 303 kB 00:00 2024-04-25T14:47:39.597 INFO:teuthology.orchestra.run.smithi094.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 402 kB/s | 35 kB 00:00 2024-04-25T14:47:39.631 INFO:teuthology.orchestra.run.smithi094.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.3 MB/s | 206 kB 00:00 2024-04-25T14:47:39.747 INFO:teuthology.orchestra.run.smithi138.stdout:(148/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 2.8 MB/s | 49 MB 00:17 2024-04-25T14:47:39.781 INFO:teuthology.orchestra.run.smithi094.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-25T14:47:39.806 INFO:teuthology.orchestra.run.smithi138.stdout:(149/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.5 MB/s | 3.4 MB 00:00 2024-04-25T14:47:39.806 INFO:teuthology.orchestra.run.smithi094.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 702 kB/s | 144 kB 00:00 2024-04-25T14:47:39.831 INFO:teuthology.orchestra.run.smithi094.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 54 kB/s | 11 kB 00:00 2024-04-25T14:47:39.848 INFO:teuthology.orchestra.run.smithi138.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-04-25T14:47:39.882 INFO:teuthology.orchestra.run.smithi094.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 363 kB/s | 18 kB 00:00 2024-04-25T14:47:39.890 INFO:teuthology.orchestra.run.smithi138.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 30 MB/s | 2.4 MB 00:00 2024-04-25T14:47:39.907 INFO:teuthology.orchestra.run.smithi094.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 315 kB/s | 23 kB 00:00 2024-04-25T14:47:39.932 INFO:teuthology.orchestra.run.smithi094.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 197 kB/s | 19 kB 00:00 2024-04-25T14:47:39.940 INFO:teuthology.orchestra.run.smithi138.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 2.9 MB/s | 3.1 MB 00:01 2024-04-25T14:47:39.945 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:47:39.945 INFO:teuthology.orchestra.run.smithi138.stdout:Total 8.2 MB/s | 222 MB 00:26 2024-04-25T14:47:39.969 INFO:teuthology.orchestra.run.smithi137.stdout:(111/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 2.8 MB/s | 49 MB 00:17 2024-04-25T14:47:39.983 INFO:teuthology.orchestra.run.smithi094.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 404 kB/s | 20 kB 00:00 2024-04-25T14:47:39.993 INFO:teuthology.orchestra.run.smithi137.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 318 kB/s | 151 kB 00:00 2024-04-25T14:47:40.008 INFO:teuthology.orchestra.run.smithi094.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 629 kB/s | 46 kB 00:00 2024-04-25T14:47:40.019 INFO:teuthology.orchestra.run.smithi137.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-25T14:47:40.044 INFO:teuthology.orchestra.run.smithi137.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 7.4 MB/s | 548 kB 00:00 2024-04-25T14:47:40.058 INFO:teuthology.orchestra.run.smithi094.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-25T14:47:40.069 INFO:teuthology.orchestra.run.smithi137.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 889 kB/s | 43 kB 00:00 2024-04-25T14:47:40.083 INFO:teuthology.orchestra.run.smithi094.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 790 kB/s | 79 kB 00:00 2024-04-25T14:47:40.094 INFO:teuthology.orchestra.run.smithi137.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 435 kB/s | 32 kB 00:00 2024-04-25T14:47:40.120 INFO:teuthology.orchestra.run.smithi137.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-04-25T14:47:40.162 INFO:teuthology.orchestra.run.smithi137.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-25T14:47:40.187 INFO:teuthology.orchestra.run.smithi137.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-04-25T14:47:40.212 INFO:teuthology.orchestra.run.smithi137.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-25T14:47:40.226 INFO:teuthology.orchestra.run.smithi094.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 278 kB/s | 58 kB 00:00 2024-04-25T14:47:40.237 INFO:teuthology.orchestra.run.smithi137.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 477 kB/s | 35 kB 00:00 2024-04-25T14:47:40.262 INFO:teuthology.orchestra.run.smithi137.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T14:47:40.268 INFO:teuthology.orchestra.run.smithi094.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-25T14:47:40.293 INFO:teuthology.orchestra.run.smithi094.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 258 kB/s | 16 kB 00:00 2024-04-25T14:47:40.318 INFO:teuthology.orchestra.run.smithi094.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-04-25T14:47:40.335 INFO:teuthology.orchestra.run.smithi094.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-25T14:47:40.361 INFO:teuthology.orchestra.run.smithi094.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 753 kB/s | 31 kB 00:00 2024-04-25T14:47:40.394 INFO:teuthology.orchestra.run.smithi094.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-25T14:47:40.419 INFO:teuthology.orchestra.run.smithi094.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 710 kB/s | 59 kB 00:00 2024-04-25T14:47:40.445 INFO:teuthology.orchestra.run.smithi094.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 729 kB/s | 36 kB 00:00 2024-04-25T14:47:40.470 INFO:teuthology.orchestra.run.smithi094.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 689 kB/s | 51 kB 00:00 2024-04-25T14:47:40.496 INFO:teuthology.orchestra.run.smithi137.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-04-25T14:47:40.503 INFO:teuthology.orchestra.run.smithi094.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-25T14:47:40.513 INFO:teuthology.orchestra.run.smithi137.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 525 kB/s | 144 kB 00:00 2024-04-25T14:47:40.529 INFO:teuthology.orchestra.run.smithi094.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.6 MB/s | 90 kB 00:00 2024-04-25T14:47:40.530 INFO:teuthology.orchestra.run.smithi137.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 40 kB/s | 11 kB 00:00 2024-04-25T14:47:40.555 INFO:teuthology.orchestra.run.smithi137.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-25T14:47:40.571 INFO:teuthology.orchestra.run.smithi094.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.3 MB/s | 427 kB 00:00 2024-04-25T14:47:40.572 INFO:teuthology.orchestra.run.smithi137.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-04-25T14:47:40.589 INFO:teuthology.orchestra.run.smithi137.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-04-25T14:47:40.596 INFO:teuthology.orchestra.run.smithi094.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 229 kB/s | 20 kB 00:00 2024-04-25T14:47:40.606 INFO:teuthology.orchestra.run.smithi137.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-25T14:47:40.630 INFO:teuthology.orchestra.run.smithi094.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 3.3 MB/s | 192 kB 00:00 2024-04-25T14:47:40.715 INFO:teuthology.orchestra.run.smithi137.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-25T14:47:40.722 INFO:teuthology.orchestra.run.smithi094.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T14:47:40.731 INFO:teuthology.orchestra.run.smithi137.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2024-04-25T14:47:40.757 INFO:teuthology.orchestra.run.smithi137.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 524 kB/s | 79 kB 00:00 2024-04-25T14:47:40.774 INFO:teuthology.orchestra.run.smithi137.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-04-25T14:47:40.797 INFO:teuthology.orchestra.run.smithi094.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T14:47:40.882 INFO:teuthology.orchestra.run.smithi137.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T14:47:40.889 INFO:teuthology.orchestra.run.smithi094.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T14:47:40.899 INFO:teuthology.orchestra.run.smithi137.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-04-25T14:47:40.924 INFO:teuthology.orchestra.run.smithi137.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 598 kB/s | 90 kB 00:00 2024-04-25T14:47:40.950 INFO:teuthology.orchestra.run.smithi137.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 423 kB/s | 28 kB 00:00 2024-04-25T14:47:40.966 INFO:teuthology.orchestra.run.smithi137.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 737 kB/s | 31 kB 00:00 2024-04-25T14:47:40.992 INFO:teuthology.orchestra.run.smithi137.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-25T14:47:40.997 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T14:47:41.009 INFO:teuthology.orchestra.run.smithi137.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T14:47:41.026 INFO:teuthology.orchestra.run.smithi137.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 609 kB/s | 36 kB 00:00 2024-04-25T14:47:41.043 INFO:teuthology.orchestra.run.smithi137.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T14:47:41.135 INFO:teuthology.orchestra.run.smithi137.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T14:47:41.139 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T14:47:41.139 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T14:47:41.276 INFO:teuthology.orchestra.run.smithi137.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 358 kB/s | 90 kB 00:00 2024-04-25T14:47:41.306 INFO:teuthology.orchestra.run.smithi094.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 4.8 MB/s | 3.4 MB 00:00 2024-04-25T14:47:41.440 INFO:teuthology.orchestra.run.smithi094.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.9 MB/s | 3.1 MB 00:00 2024-04-25T14:47:41.452 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:47:41.453 INFO:teuthology.orchestra.run.smithi094.stdout:Total 8.8 MB/s | 222 MB 00:25 2024-04-25T14:47:41.468 INFO:teuthology.orchestra.run.smithi137.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.0 MB/s | 427 kB 00:00 2024-04-25T14:47:41.493 INFO:teuthology.orchestra.run.smithi137.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 56 kB/s | 20 kB 00:00 2024-04-25T14:47:41.519 INFO:teuthology.orchestra.run.smithi137.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 791 kB/s | 192 kB 00:00 2024-04-25T14:47:41.561 INFO:teuthology.orchestra.run.smithi137.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T14:47:41.628 INFO:teuthology.orchestra.run.smithi137.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T14:47:41.736 INFO:teuthology.orchestra.run.smithi137.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-04-25T14:47:42.136 INFO:teuthology.orchestra.run.smithi137.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.5 MB/s | 3.4 MB 00:00 2024-04-25T14:47:42.428 INFO:teuthology.orchestra.run.smithi137.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.5 MB/s | 3.1 MB 00:00 2024-04-25T14:47:42.436 INFO:teuthology.orchestra.run.smithi137.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:47:42.436 INFO:teuthology.orchestra.run.smithi137.stdout:Total 7.9 MB/s | 222 MB 00:27 2024-04-25T14:47:42.891 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T14:47:42.891 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T14:47:43.564 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction check 2024-04-25T14:47:43.710 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction check succeeded. 2024-04-25T14:47:43.711 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction test 2024-04-25T14:47:44.379 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-04-25T14:47:44.641 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-04-25T14:47:44.641 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-04-25T14:47:45.468 INFO:teuthology.orchestra.run.smithi137.stdout:Transaction test succeeded. 2024-04-25T14:47:45.468 INFO:teuthology.orchestra.run.smithi137.stdout:Running transaction 2024-04-25T14:47:47.307 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-04-25T14:47:47.307 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-04-25T14:47:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T14:47:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:47:49.165 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:47:49.227 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:47:49.261 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:47:49.287 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:47:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:47:49.648 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:47:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:47:49.884 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:47:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:47:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:47:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:47:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:47:50.198 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:47:50.226 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:47:50.329 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:47:50.352 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:47:50.415 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:47:50.437 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:47:50.454 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:47:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:47:50.653 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:47:50.763 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:47:50.855 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:47:50.886 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:47:50.921 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:47:50.986 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:47:51.029 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:47:51.072 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:47:51.113 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:47:51.141 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:47:51.209 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:47:51.262 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:47:51.294 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:47:51.338 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:47:51.414 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:47:51.448 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:47:51.456 INFO:teuthology.orchestra.run.smithi137.stdout: Preparing : 1/1 2024-04-25T14:47:51.483 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:47:51.512 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:47:51.531 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:47:51.570 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:47:51.581 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:47:51.623 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:47:51.627 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:47:51.657 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:47:51.683 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:47:51.717 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:47:51.725 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:47:51.851 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:47:51.912 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:47:52.092 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:47:52.209 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:47:52.358 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:47:52.394 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:47:52.425 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:47:52.469 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:47:52.507 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:47:52.515 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:47:52.546 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:47:52.580 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:47:52.618 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:47:52.659 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:47:52.661 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:47:52.712 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:47:52.740 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:47:52.826 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:47:52.836 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:47:52.866 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:47:52.896 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:47:52.929 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:47:52.950 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:47:52.950 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:47:52.971 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:47:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:47:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:47:53.083 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:47:53.129 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:47:53.159 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:47:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:47:53.191 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:47:53.209 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:47:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:47:53.299 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:47:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:47:53.395 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:47:53.424 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:47:53.431 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:47:53.458 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:47:53.462 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:47:53.522 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:47:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:47:53.557 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:47:53.609 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:47:53.621 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:47:53.624 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-04-25T14:47:53.642 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:47:53.670 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:47:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:47:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:47:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:47:53.739 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:47:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T14:47:53.760 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:47:53.817 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:47:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:47:53.847 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:47:53.853 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T14:47:53.883 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:47:53.959 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:47:53.961 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:47:53.987 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T14:47:53.993 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:47:53.994 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:47:54.021 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:47:54.037 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:47:54.048 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T14:47:54.057 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:47:54.064 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:47:54.083 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T14:47:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:47:54.127 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:47:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:47:54.164 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:47:54.172 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:47:54.199 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:47:54.232 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:47:54.233 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T14:47:54.245 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:47:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:47:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:47:54.380 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:47:54.394 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:47:54.438 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:47:54.470 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:47:54.475 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:47:54.517 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:47:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:47:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:47:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T14:47:54.773 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:47:54.940 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:47:54.966 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:47:54.999 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:47:55.078 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:47:55.094 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:47:55.107 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:47:55.141 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:47:55.150 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:47:55.188 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:47:55.230 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:47:55.378 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:47:55.385 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:47:55.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:47:55.441 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:47:55.461 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:47:55.481 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T14:47:55.496 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:47:55.551 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T14:47:55.558 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:47:55.568 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:47:55.579 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:47:55.600 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:47:55.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:47:55.636 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:47:55.683 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:47:55.696 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:47:55.727 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:47:55.733 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:47:55.746 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:47:55.763 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:47:55.770 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:47:55.812 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:47:55.850 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T14:47:55.882 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:47:55.924 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:47:55.949 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:47:55.952 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:47:55.955 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T14:47:55.993 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:47:56.023 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:47:56.029 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:47:56.065 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:47:56.159 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:47:56.223 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:47:56.255 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:47:56.280 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:47:56.281 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:47:56.305 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:47:56.342 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T14:47:56.380 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:47:56.507 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:47:56.539 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T14:47:56.546 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:47:56.573 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:47:56.608 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:47:56.647 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:47:56.659 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T14:47:56.664 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:47:56.678 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:47:56.700 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T14:47:56.709 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:47:56.734 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:47:56.747 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T14:47:56.768 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:47:56.774 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:47:56.809 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:47:56.844 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:47:56.850 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T14:47:56.914 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T14:47:56.930 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:47:56.974 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:47:56.976 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T14:47:57.005 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:47:57.047 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T14:47:57.053 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:47:57.091 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T14:47:57.150 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:47:57.206 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:47:57.212 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T14:47:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T14:47:57.318 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T14:47:57.378 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T14:47:57.483 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T14:47:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T14:47:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T14:47:57.620 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T14:47:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T14:47:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T14:47:57.895 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T14:47:57.918 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:47:57.963 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:47:57.981 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:47:57.996 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:47:58.069 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:47:58.083 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T14:47:58.094 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:47:58.154 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:47:58.160 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T14:47:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:47:58.240 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:47:58.315 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:47:58.410 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:47:58.493 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:47:58.529 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:47:58.606 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:47:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:47:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:47:58.824 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T14:47:58.889 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:47:58.919 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:47:58.924 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T14:47:58.956 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:47:59.009 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T14:47:59.058 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T14:47:59.108 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T14:47:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T14:47:59.234 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:47:59.351 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:47:59.428 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:47:59.436 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T14:47:59.468 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:47:59.539 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:47:59.550 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T14:47:59.561 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:47:59.633 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T14:47:59.731 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T14:47:59.774 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T14:47:59.814 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T14:47:59.883 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T14:47:59.930 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T14:48:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T14:48:00.096 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T14:48:00.304 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T14:48:00.353 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T14:48:00.401 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T14:48:00.458 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T14:48:00.562 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:48:00.598 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T14:48:00.642 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T14:48:00.642 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:48:00.669 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:48:00.697 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T14:48:00.702 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:48:00.733 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T14:48:00.766 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T14:48:00.959 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T14:48:01.014 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:48:01.090 INFO:teuthology.orchestra.run.smithi137.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:48:01.141 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T14:48:01.181 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:48:01.181 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T14:48:01.203 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:48:01.227 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T14:48:01.248 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:48:01.274 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T14:48:01.332 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T14:48:01.384 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T14:48:01.426 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T14:48:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T14:48:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T14:48:01.598 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T14:48:01.646 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T14:48:01.724 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:48:01.760 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:48:01.786 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T14:48:01.831 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:48:01.851 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:01.853 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T14:48:01.861 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:48:01.891 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:01.922 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T14:48:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T14:48:02.151 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T14:48:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T14:48:02.445 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:48:02.571 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:48:02.571 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:48:02.571 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:48:02.571 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:02.605 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:48:03.156 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T14:48:03.210 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T14:48:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T14:48:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T14:48:03.463 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T14:48:03.561 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T14:48:03.596 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T14:48:03.826 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T14:48:03.860 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T14:48:03.964 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T14:48:04.116 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:04.166 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:04.711 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:04.745 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:48:04.784 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T14:48:04.836 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:48:04.836 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:48:04.836 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:48:04.836 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:04.862 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:48:04.934 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T14:48:06.499 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T14:48:06.604 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T14:48:06.650 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T14:48:06.715 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T14:48:07.107 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T14:48:07.158 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:48:07.283 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T14:48:07.315 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T14:48:07.372 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T14:48:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T14:48:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:48:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T14:48:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T14:48:10.779 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:10.868 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T14:48:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:48:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T14:48:11.749 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T14:48:11.749 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T14:48:11.749 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:11.792 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:48:14.370 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:48:14.370 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-25T14:48:14.371 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-25T14:48:14.371 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-25T14:48:14.371 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-25T14:48:14.371 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-25T14:48:14.371 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-04-25T14:48:14.371 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-25T14:48:14.371 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:14.544 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:48:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:48:14.559 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:14.559 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:48:14.559 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:16.047 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:48:16.072 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:48:16.072 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:16.072 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:48:16.072 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:48:16.072 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:48:16.072 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:16.245 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:48:16.245 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /sys 2024-04-25T14:48:16.245 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /proc 2024-04-25T14:48:16.245 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /mnt 2024-04-25T14:48:16.245 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /var/tmp 2024-04-25T14:48:16.246 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /home 2024-04-25T14:48:16.246 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /root 2024-04-25T14:48:16.246 INFO:teuthology.orchestra.run.smithi137.stdout:skipping the directory /tmp 2024-04-25T14:48:16.246 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:16.306 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:48:16.328 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:48:16.328 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:16.328 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:48:16.328 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:48:16.328 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:48:16.328 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:16.417 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:48:16.434 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:48:16.434 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:16.434 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:48:16.434 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:48:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:48:16.787 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:16.787 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:48:16.787 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:48:16.787 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:48:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:48:16.942 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:48:16.998 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:48:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:48:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:48:17.292 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:48:17.388 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:48:17.518 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:48:17.557 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:48:17.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:48:17.648 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:48:17.919 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:48:17.945 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:48:17.945 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:17.945 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:48:17.945 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:48:17.945 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:48:17.946 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:17.994 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:48:18.025 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:48:18.056 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:48:18.117 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:48:18.156 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:48:18.162 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:48:18.184 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:48:18.184 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:18.184 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:48:18.184 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:48:18.184 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:48:18.184 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:18.621 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:48:18.643 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:48:18.643 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:18.644 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:48:18.644 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:48:18.644 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:48:18.644 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:18.751 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:48:18.808 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:48:18.862 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:48:19.049 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:48:19.096 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:48:19.175 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:48:19.182 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:48:19.216 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:48:19.271 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:48:19.412 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:48:19.455 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:48:19.478 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:48:19.506 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:48:19.983 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:48:20.013 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:48:20.045 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:48:20.107 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:48:20.144 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:48:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:48:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:48:20.317 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:48:20.488 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:48:20.519 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:48:20.541 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:48:20.541 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:20.541 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:48:20.542 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:48:20.542 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:48:20.542 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:20.586 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:48:20.597 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:48:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:48:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:48:21.177 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:48:21.213 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:48:21.666 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:48:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:48:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:48:21.719 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:21.719 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:48:21.719 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:48:21.719 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:48:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:48:21.794 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:48:21.794 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:21.794 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:48:21.794 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:21.840 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:48:21.862 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:48:21.862 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:21.862 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:48:21.862 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:48:21.862 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:48:21.862 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:22.166 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:48:22.179 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:48:22.188 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:48:22.188 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:22.189 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:48:22.189 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:48:22.189 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:48:22.189 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:22.205 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:48:22.307 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:48:22.476 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:48:22.515 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:48:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:48:22.539 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:22.539 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:48:22.539 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:48:22.539 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:48:22.539 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:22.591 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:48:22.603 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:48:22.700 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:48:22.756 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:48:23.681 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:48:23.710 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:48:23.733 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:48:23.733 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:23.734 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:48:23.734 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:48:23.734 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:48:23.734 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:23.777 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:48:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:48:23.800 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:23.800 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:48:23.800 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:23.853 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:48:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:48:23.876 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:23.876 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:48:23.876 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:48:23.876 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:48:23.876 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:24.179 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:48:24.201 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:48:24.202 INFO:teuthology.orchestra.run.smithi137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:24.202 INFO:teuthology.orchestra.run.smithi137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:48:24.202 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:48:24.202 INFO:teuthology.orchestra.run.smithi137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:48:24.202 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-04-25T14:48:24.774 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:25.014 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:48:25.062 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T14:48:25.062 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:25.062 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T14:48:25.062 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:48:26.753 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:48:26.785 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T14:48:26.785 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:26.785 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T14:48:26.785 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:48:26.786 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T14:48:26.786 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:26.794 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:48:26.835 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:48:26.867 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:48:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:48:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:48:27.093 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:48:27.139 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T14:48:27.139 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:27.139 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T14:48:27.139 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:48:27.139 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T14:48:27.139 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:48:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:48:27.635 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:48:27.704 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T14:48:27.704 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:27.704 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T14:48:27.704 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:48:27.704 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T14:48:27.704 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:27.775 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:27.819 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:27.836 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T14:48:27.838 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T14:48:27.998 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T14:48:28.231 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T14:48:28.290 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T14:48:28.375 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T14:48:28.484 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:48:28.649 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:48:28.664 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:48:28.691 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T14:48:28.730 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T14:48:28.741 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:48:28.773 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T14:48:28.782 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:48:28.799 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:48:28.840 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:48:28.875 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:48:28.881 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:48:28.913 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:48:28.922 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:48:28.922 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:48:28.941 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:48:29.015 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:48:29.048 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:48:29.048 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:48:29.122 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:48:29.137 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:48:29.170 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:48:29.204 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:29.208 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:48:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:48:29.460 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T14:48:29.515 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T14:48:29.626 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T14:48:29.663 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:48:29.787 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:29.831 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:29.851 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:30.691 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:48:30.769 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:48:30.810 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:48:30.891 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:48:30.923 INFO:teuthology.orchestra.run.smithi137.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:48:30.924 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:48:30.983 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T14:48:31.016 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:48:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:48:31.057 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:48:31.057 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:48:31.146 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:48:31.214 INFO:teuthology.orchestra.run.smithi137.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:32.134 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T14:48:32.174 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:48:32.326 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T14:48:32.513 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T14:48:32.551 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:48:32.627 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T14:48:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T14:48:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:48:32.628 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T14:48:32.628 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:32.685 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:48:32.723 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T14:48:32.857 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T14:48:32.939 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T14:48:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T14:48:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:48:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T14:48:34.102 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:34.102 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T14:48:34.102 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:48:34.102 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T14:48:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:34.148 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:48:34.233 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T14:48:34.233 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:34.233 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T14:48:34.233 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:34.298 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:48:34.398 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T14:48:34.398 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:34.398 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T14:48:34.398 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:48:34.398 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T14:48:34.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:34.791 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:48:34.878 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T14:48:34.879 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T14:48:34.879 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T14:48:34.879 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:48:34.879 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T14:48:34.879 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:48:35.418 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:48:35.419 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:48:35.420 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:48:35.420 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:48:35.420 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:48:35.420 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:48:35.420 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:48:35.420 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:48:35.420 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:48:35.432 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:48:35.433 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:48:35.434 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:48:35.435 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:48:35.436 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:48:35.437 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:48:35.438 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:48:35.439 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:48:35.440 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:48:37.270 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:48:37.271 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:48:37.272 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:48:37.272 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:48:37.283 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:48:37.283 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:48:37.283 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:48:37.283 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:48:37.283 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:48:37.283 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:48:37.284 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:48:37.285 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:48:37.286 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:48:37.287 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:48:37.288 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:48:37.289 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:48:37.290 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:48:37.291 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:48:37.292 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:48:37.540 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:37.540 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:37.540 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.541 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.542 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:48:37.543 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:48:37.544 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:48:37.545 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:48:37.546 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:48:37.547 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:48:37.548 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:48:37.549 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:48:37.550 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:48:37.551 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:48:37.551 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:48:37.551 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:48:37.551 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:48:37.551 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:48:37.551 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T14:48:37.929 DEBUG:teuthology.parallel:result is None 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout:Upgraded: 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout:Installed: 2024-04-25T14:48:39.324 INFO:teuthology.orchestra.run.smithi137.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.325 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:39.326 INFO:teuthology.orchestra.run.smithi137.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.327 INFO:teuthology.orchestra.run.smithi137.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:48:39.328 INFO:teuthology.orchestra.run.smithi137.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:48:39.329 INFO:teuthology.orchestra.run.smithi137.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:48:39.330 INFO:teuthology.orchestra.run.smithi137.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:48:39.331 INFO:teuthology.orchestra.run.smithi137.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:48:39.332 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:48:39.333 INFO:teuthology.orchestra.run.smithi137.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.334 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:48:39.335 INFO:teuthology.orchestra.run.smithi137.stdout: 2024-04-25T14:48:39.336 INFO:teuthology.orchestra.run.smithi137.stdout:Complete! 2024-04-25T14:48:39.697 DEBUG:teuthology.parallel:result is None 2024-04-25T14:48:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T14:48:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T14:48:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T14:48:40.071 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T14:48:40.119 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T14:48:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T14:48:40.507 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T14:48:40.535 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T14:48:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:41.417 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:41.479 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T14:48:42.836 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T14:48:42.951 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T14:48:43.036 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T14:48:43.203 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T14:48:43.265 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T14:48:43.265 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:48:43.383 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T14:48:43.423 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:48:43.423 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:48:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T14:48:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:44.004 INFO:teuthology.orchestra.run.smithi094.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T14:48:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T14:48:49.892 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T14:48:49.893 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T14:48:49.894 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T14:48:49.894 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T14:48:49.894 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T14:48:49.894 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T14:48:49.905 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T14:48:49.906 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T14:48:49.907 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T14:48:49.908 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T14:48:49.909 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T14:48:49.910 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T14:48:49.911 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T14:48:49.912 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T14:48:49.913 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T14:48:49.914 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T14:48:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T14:48:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:52.275 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T14:48:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T14:48:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T14:48:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T14:48:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T14:48:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T14:48:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T14:48:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T14:48:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T14:48:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T14:48:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T14:48:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:48:52.288 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-04-25T14:48:52.903 DEBUG:teuthology.parallel:result is None 2024-04-25T14:48:52.904 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:48:53.053 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:48:53.110 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:48:53.110 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:48:53.110 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:48:53.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:48:53.248 DEBUG:teuthology.orchestra.run.smithi137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:48:53.274 INFO:teuthology.orchestra.run.smithi137.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:48:53.274 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:48:53.274 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:48:53.276 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:48:53.417 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T14:48:53.442 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T14:48:53.442 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T14:48:53.443 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T14:48:53.445 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T14:48:53.445 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:48:53.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:48:53.522 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:48:53.522 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:48:53.553 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:48:53.553 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T14:48:53.583 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T14:48:53.583 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:48:53.583 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:48:53.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:48:53.759 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:48:53.760 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:48:53.788 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:48:53.855 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:48:53.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T14:48:53.884 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T14:48:53.950 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T14:48:53.951 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:48:53.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:48:54.029 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:48:54.135 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:48:54.135 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:48:54.160 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:48:54.224 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:48:54.224 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T14:48:54.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T14:48:54.312 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T14:48:54.312 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:48:54.312 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:48:54.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:48:54.490 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:48:54.491 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:48:54.515 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:48:54.580 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:48:54.580 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T14:48:54.604 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T14:48:54.668 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T14:48:54.768 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T14:48:54.768 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:48:54.768 INFO:tasks.cephadm:Cluster fsid is f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:48:54.768 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T14:48:54.768 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.94', 'mon.b': '172.21.15.137', 'mon.c': '172.21.15.138'} 2024-04-25T14:48:54.768 INFO:tasks.cephadm:First mon is mon.a on smithi094 2024-04-25T14:48:54.769 INFO:tasks.cephadm:First mgr is x 2024-04-25T14:48:54.769 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T14:48:54.769 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-04-25T14:48:54.838 DEBUG:teuthology.orchestra.run.smithi137:> sudo hostname $(hostname -s) 2024-04-25T14:48:54.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-04-25T14:48:54.885 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T14:48:54.885 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:48:55.102 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T14:48:55.238 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:48:55.239 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T14:48:55.239 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:55.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:48 /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:55.985 DEBUG:teuthology.orchestra.run.smithi137:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:56.695 INFO:teuthology.orchestra.run.smithi137.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:48 /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:56.695 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:57.424 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 14:48 /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:57.425 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:57.469 DEBUG:teuthology.orchestra.run.smithi137:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:57.488 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T14:48:57.525 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T14:48:57.525 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:48:57.530 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:48:57.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T14:48:59.131 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:48:59.196 INFO:teuthology.orchestra.run.smithi137.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:48:59.723 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:50:24.678 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T14:50:24.679 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:50:24.679 INFO:teuthology.orchestra.run.smithi137.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:50:24.679 INFO:teuthology.orchestra.run.smithi137.stdout: "repo_digests": [ 2024-04-25T14:50:24.679 INFO:teuthology.orchestra.run.smithi137.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:50:24.679 INFO:teuthology.orchestra.run.smithi137.stdout: ] 2024-04-25T14:50:24.679 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T14:50:27.543 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T14:50:27.543 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:50:27.544 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:50:27.544 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-04-25T14:50:27.544 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:50:27.544 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-25T14:50:27.544 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T14:50:33.120 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T14:50:33.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T14:50:33.120 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T14:50:33.120 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-04-25T14:50:33.120 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T14:50:33.120 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T14:50:33.120 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T14:50:33.158 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-04-25T14:50:33.245 DEBUG:teuthology.orchestra.run.smithi137:> sudo mkdir -p /etc/ceph 2024-04-25T14:50:33.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-04-25T14:50:33.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-04-25T14:50:33.395 DEBUG:teuthology.orchestra.run.smithi137:> sudo chmod 777 /etc/ceph 2024-04-25T14:50:33.423 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-04-25T14:50:33.450 INFO:tasks.cephadm:Writing seed config... 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T14:50:33.452 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T14:50:33.453 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T14:50:33.454 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T14:50:33.455 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T14:50:33.456 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:50:33.456 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T14:50:33.511 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f04d929c-0312-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T14:50:33.512 DEBUG:teuthology.orchestra.run.smithi094:mon.a> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T14:50:33.559 DEBUG:teuthology.orchestra.run.smithi094:mgr.x> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T14:50:33.605 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T14:50:33.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid f04d929c-0312-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:50:33.879 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-04-25T14:50:33.879 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'f04d929c-0312-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-04-25T14:50:33.879 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T14:50:33.880 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-04-25T14:50:33.932 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:50:33.932 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-04-25T14:50:33.932 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-04-25T14:50:33.947 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:50:33.947 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:50:33.960 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:50:33.961 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T14:50:33.980 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-04-25T14:50:33.997 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-04-25T14:50:33.997 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:50:33.997 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-04-25T14:50:34.059 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T14:50:34.060 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T14:50:34.060 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-04-25T14:50:34.060 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-04-25T14:50:34.075 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T14:50:34.075 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T14:50:34.093 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T14:50:34.093 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T14:50:34.119 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-04-25T14:50:34.141 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-04-25T14:50:34.141 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-04-25T14:50:34.141 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-04-25T14:50:34.141 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:34.142 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 140624530611360 on /run/cephadm/f04d929c-0312-11ef-bc93-c7b262605968.lock 2024-04-25T14:50:34.142 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 140624530611360 acquired on /run/cephadm/f04d929c-0312-11ef-bc93-c7b262605968.lock 2024-04-25T14:50:34.144 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-04-25T14:50:34.145 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-04-25T14:50:34.145 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-04-25T14:50:34.160 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-04-25T14:50:34.160 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-04-25T14:50:34.172 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T14:50:34.172 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-25T14:50:34.172 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1747sec hoplimit 64 pref medium 2024-04-25T14:50:34.183 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T14:50:34.183 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T14:50:34.183 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:50:34.183 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T14:50:34.183 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-25T14:50:34.184 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T14:50:34.186 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-04-25T14:50:34.186 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-04-25T14:50:34.186 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T14:50:34.189 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T14:50:34.191 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:50:34.915 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T14:50:34.915 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T14:50:34.915 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T14:50:34.915 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T14:50:34.915 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T14:50:34.916 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T14:50:34.916 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T14:50:36.159 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T14:50:36.159 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T14:50:36.159 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T14:50:37.378 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-04-25T14:50:37.379 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-04-25T14:50:38.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQC9bSpm8x7zMRAAzNA0G2zmP8n8o9s/DmCL3Q== 2024-04-25T14:50:39.626 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQC+bSpmrYAnOhAAyHwW0AFdytqfFeiQKQ4C6g== 2024-04-25T14:50:40.807 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDAbSpmlvKCCBAADKe8q5GpS91Gim8tru9Arw== 2024-04-25T14:50:40.810 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-04-25T14:50:41.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for a [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T14:50:41.925 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:50:41.926 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-04-25T14:50:43.084 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.a on 2024-04-25T14:50:43.448 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T14:50:43.688 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T14:50:43.997 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f04d929c-0312-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f04d929c-0312-11ef-bc93-c7b262605968.target. 2024-04-25T14:50:43.998 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f04d929c-0312-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f04d929c-0312-11ef-bc93-c7b262605968.target. 2024-04-25T14:50:44.367 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a 2024-04-25T14:50:44.368 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T14:50:44.652 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f04d929c-0312-11ef-bc93-c7b262605968.target.wants/ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f04d929c-0312-11ef-bc93-c7b262605968@.service. 2024-04-25T14:50:45.315 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:45 smithi094 podman[27473]: 2024-04-25 14:50:45.059636301 +0000 UTC m=+0.145026003 container create 71d4bcb43650e16597ac7c1c4c9620ca9fd10cd7cdb697f333f946b8dd233186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:50:45.315 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:45 smithi094 podman[27473]: 2024-04-25 14:50:45.313308672 +0000 UTC m=+0.398698387 container init 71d4bcb43650e16597ac7c1c4c9620ca9fd10cd7cdb697f333f946b8dd233186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:50:45.465 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T14:50:45.484 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:50:45.484 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T14:50:45.484 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T14:50:45.484 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:50:45.485 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-04-25T14:50:45.485 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-04-25T14:50:45.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:45 smithi094 podman[27473]: 2024-04-25 14:50:45.324025608 +0000 UTC m=+0.409415317 container start 71d4bcb43650e16597ac7c1c4c9620ca9fd10cd7cdb697f333f946b8dd233186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:50:45.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:45 smithi094 ceph-mon[27492]: mkfs f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:45.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:45 smithi094 ceph-mon[27492]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:50:45.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:45 smithi094 bash[27473]: 71d4bcb43650e16597ac7c1c4c9620ca9fd10cd7cdb697f333f946b8dd233186 2024-04-25T14:50:45.633 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:45 smithi094 systemd[1]: Started Ceph mon.a for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: monmap epoch 1 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: last_changed 2024-04-25T14:50:41.313017+0000 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: min_mon_release 19 (squid) 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: election_strategy: 1 2024-04-25T14:50:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:50:46.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: fsmap 2024-04-25T14:50:46.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:50:46.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: mgrmap e1: no daemons active 2024-04-25T14:50:46.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:46 smithi094 ceph-mon[27492]: from='client.? 172.21.15.94:0/3522598095' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T14:50:46.827 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.755972s) 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.063+0000 7f1ed4400700 1 Processor -- start 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.063+0000 7f1ed4400700 1 -- start start 2024-04-25T14:50:46.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.064+0000 7f1ed4400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.064+0000 7f1ed4400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc108fd0 con 0x7f1ecc1085e0 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.065+0000 7f1ed1a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.065+0000 7f1ed1a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34496/0 (socket says 172.21.15.94:34496) 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.065+0000 7f1ed1a00700 1 -- 172.21.15.94:0/927546311 learned_addr learned my addr 172.21.15.94:0/927546311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.065+0000 7f1ed1a00700 1 -- 172.21.15.94:0/927546311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc109790 con 0x7f1ecc1085e0 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.066+0000 7f1ed1a00700 1 --2- 172.21.15.94:0/927546311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc108a00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00098a0 tx=0x7f1ec0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe66d41a286caa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.066+0000 7f1ecbe00700 1 -- 172.21.15.94:0/927546311 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec0004160 con 0x7f1ecc1085e0 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.066+0000 7f1ecbe00700 1 -- 172.21.15.94:0/927546311 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0004300 con 0x7f1ecc1085e0 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.066+0000 7f1ecbe00700 1 -- 172.21.15.94:0/927546311 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec00036a0 con 0x7f1ecc1085e0 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.067+0000 7f1ed4400700 1 -- 172.21.15.94:0/927546311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 msgr2=0x7f1ecc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.067+0000 7f1ed4400700 1 --2- 172.21.15.94:0/927546311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc108a00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00098a0 tx=0x7f1ec0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.067+0000 7f1ed4400700 1 -- 172.21.15.94:0/927546311 shutdown_connections 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.067+0000 7f1ed4400700 1 --2- 172.21.15.94:0/927546311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc108a00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.067+0000 7f1ed4400700 1 -- 172.21.15.94:0/927546311 >> 172.21.15.94:0/927546311 conn(0x7f1ecc103c60 msgr2=0x7f1ecc106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:46.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.067+0000 7f1ed4400700 1 -- 172.21.15.94:0/927546311 shutdown_connections 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.067+0000 7f1ed4400700 1 -- 172.21.15.94:0/927546311 wait complete. 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.068+0000 7f1ed4400700 1 Processor -- start 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.068+0000 7f1ed4400700 1 -- start start 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.068+0000 7f1ed4400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.069+0000 7f1ed4400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ecc109cc0 con 0x7f1ecc1085e0 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.069+0000 7f1ed1a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.069+0000 7f1ed1a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34504/0 (socket says 172.21.15.94:34504) 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.069+0000 7f1ed1a00700 1 -- 172.21.15.94:0/3522598095 learned_addr learned my addr 172.21.15.94:0/3522598095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.069+0000 7f1ed1a00700 1 -- 172.21.15.94:0/3522598095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ecc19c180 con 0x7f1ecc1085e0 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.069+0000 7f1ed1a00700 1 --2- 172.21.15.94:0/3522598095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc19bc40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000c00 tx=0x7f1ec0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.070+0000 7f1eca000700 1 -- 172.21.15.94:0/3522598095 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec0003db0 con 0x7f1ecc1085e0 2024-04-25T14:50:46.830 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.070+0000 7f1eca000700 1 -- 172.21.15.94:0/3522598095 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec0011630 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.070+0000 7f1eca000700 1 -- 172.21.15.94:0/3522598095 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ec0011950 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.070+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ecc19c410 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.070+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ecc19f100 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.070+0000 7f1eca000700 1 -- 172.21.15.94:0/3522598095 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ec001a460 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.070+0000 7f1eca000700 1 -- 172.21.15.94:0/3522598095 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1ec001b780 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.071+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ecc0640a0 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.074+0000 7f1eca000700 1 -- 172.21.15.94:0/3522598095 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1ec001b980 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.153+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f1ecc02e080 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.154+0000 7f1eca000700 1 -- 172.21.15.94:0/3522598095 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f1ec001bb60 con 0x7f1ecc1085e0 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.155+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 msgr2=0x7f1ecc19bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.155+0000 7f1ed4400700 1 --2- 172.21.15.94:0/3522598095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc19bc40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0000c00 tx=0x7f1ec0004750 comp rx=0 tx=0).stop 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.156+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 shutdown_connections 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.156+0000 7f1ed4400700 1 --2- 172.21.15.94:0/3522598095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ecc1085e0 0x7f1ecc19bc40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.156+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 >> 172.21.15.94:0/3522598095 conn(0x7f1ecc103c60 msgr2=0x7f1ecc192940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:46.831 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.156+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 shutdown_connections 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:46.156+0000 7f1ed4400700 1 -- 172.21.15.94:0/3522598095 wait complete. 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-04-25T14:50:46.832 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T14:50:47.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:47 smithi094 ceph-mon[27492]: from='client.? 172.21.15.94:0/1520308094' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:50:47.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:47 smithi094 ceph-mon[27492]: from='client.? 172.21.15.94:0/1520308094' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:50:48.143 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.414+0000 7f4e74e00700 1 Processor -- start 2024-04-25T14:50:48.144 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.415+0000 7f4e74e00700 1 -- start start 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.415+0000 7f4e74e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e70108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.415+0000 7f4e74e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70108fd0 con 0x7f4e701085e0 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.416+0000 7f4e6de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e70108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.416+0000 7f4e6de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e70108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57328/0 (socket says 172.21.15.94:57328) 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.416+0000 7f4e6de00700 1 -- 172.21.15.94:0/1960694468 learned_addr learned my addr 172.21.15.94:0/1960694468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.416+0000 7f4e6de00700 1 -- 172.21.15.94:0/1960694468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e70109790 con 0x7f4e701085e0 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.417+0000 7f4e6de00700 1 --2- 172.21.15.94:0/1960694468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e70108a00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4e600098a0 tx=0x7f4e60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8dc816a26372e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.417+0000 7f4e6d400700 1 -- 172.21.15.94:0/1960694468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e60004160 con 0x7f4e701085e0 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.417+0000 7f4e6d400700 1 -- 172.21.15.94:0/1960694468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e60004300 con 0x7f4e701085e0 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.417+0000 7f4e6d400700 1 -- 172.21.15.94:0/1960694468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e600036a0 con 0x7f4e701085e0 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.418+0000 7f4e74e00700 1 -- 172.21.15.94:0/1960694468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 msgr2=0x7f4e70108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.418+0000 7f4e74e00700 1 --2- 172.21.15.94:0/1960694468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e70108a00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4e600098a0 tx=0x7f4e60009bb0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.418+0000 7f4e74e00700 1 -- 172.21.15.94:0/1960694468 shutdown_connections 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.418+0000 7f4e74e00700 1 --2- 172.21.15.94:0/1960694468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e70108a00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.418+0000 7f4e74e00700 1 -- 172.21.15.94:0/1960694468 >> 172.21.15.94:0/1960694468 conn(0x7f4e70103c00 msgr2=0x7f4e70106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:48.145 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.418+0000 7f4e74e00700 1 -- 172.21.15.94:0/1960694468 shutdown_connections 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.418+0000 7f4e74e00700 1 -- 172.21.15.94:0/1960694468 wait complete. 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.419+0000 7f4e74e00700 1 Processor -- start 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.419+0000 7f4e74e00700 1 -- start start 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.419+0000 7f4e74e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e7019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.419+0000 7f4e74e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70109cc0 con 0x7f4e701085e0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.420+0000 7f4e6de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e7019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.420+0000 7f4e6de00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e7019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57336/0 (socket says 172.21.15.94:57336) 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.420+0000 7f4e6de00700 1 -- 172.21.15.94:0/1520308094 learned_addr learned my addr 172.21.15.94:0/1520308094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.420+0000 7f4e6de00700 1 -- 172.21.15.94:0/1520308094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7019e3a0 con 0x7f4e701085e0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.421+0000 7f4e6de00700 1 --2- 172.21.15.94:0/1520308094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e7019de60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4e60000c00 tx=0x7f4e60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.421+0000 7f4e67400700 1 -- 172.21.15.94:0/1520308094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e60003db0 con 0x7f4e701085e0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.421+0000 7f4e67400700 1 -- 172.21.15.94:0/1520308094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e60011630 con 0x7f4e701085e0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.421+0000 7f4e67400700 1 -- 172.21.15.94:0/1520308094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e60011950 con 0x7f4e701085e0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.421+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7019e630 con 0x7f4e701085e0 2024-04-25T14:50:48.146 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.421+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a1320 con 0x7f4e701085e0 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.422+0000 7f4e67400700 1 -- 172.21.15.94:0/1520308094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e60011af0 con 0x7f4e701085e0 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.422+0000 7f4e67400700 1 -- 172.21.15.94:0/1520308094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e6001c450 con 0x7f4e701085e0 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.423+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e700640a0 con 0x7f4e701085e0 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.425+0000 7f4e67400700 1 -- 172.21.15.94:0/1520308094 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4e60032020 con 0x7f4e701085e0 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.502+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4e701a28e0 con 0x7f4e701085e0 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.509+0000 7f4e67400700 1 -- 172.21.15.94:0/1520308094 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f4e60004260 con 0x7f4e701085e0 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.511+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 msgr2=0x7f4e7019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.511+0000 7f4e74e00700 1 --2- 172.21.15.94:0/1520308094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e7019de60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4e60000c00 tx=0x7f4e60004750 comp rx=0 tx=0).stop 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.511+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 shutdown_connections 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.511+0000 7f4e74e00700 1 --2- 172.21.15.94:0/1520308094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e701085e0 0x7f4e7019de60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.511+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 >> 172.21.15.94:0/1520308094 conn(0x7f4e70103c00 msgr2=0x7f4e7007d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.511+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 shutdown_connections 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:47.511+0000 7f4e74e00700 1 -- 172.21.15.94:0/1520308094 wait complete. 2024-04-25T14:50:48.147 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-04-25T14:50:49.166 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:48 smithi094 ceph-mon[27492]: from='client.? 172.21.15.94:0/1567813228' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:50:49.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.765+0000 7f721d600700 1 Processor -- start 2024-04-25T14:50:49.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.765+0000 7f721d600700 1 -- start start 2024-04-25T14:50:49.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.765+0000 7f721d600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f721810f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:49.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.765+0000 7f721d600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f721810fa30 con 0x7f721810f040 2024-04-25T14:50:49.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.766+0000 7f7216a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f721810f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:49.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.766+0000 7f7216a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f721810f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57342/0 (socket says 172.21.15.94:57342) 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.766+0000 7f7216a00700 1 -- 172.21.15.94:0/3586416283 learned_addr learned my addr 172.21.15.94:0/3586416283 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.766+0000 7f7216a00700 1 -- 172.21.15.94:0/3586416283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721810fdc0 con 0x7f721810f040 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.766+0000 7f7216a00700 1 --2- 172.21.15.94:0/3586416283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f721810f460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72080098a0 tx=0x7f7208009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e2b0be0bfba84a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.767+0000 7f7215600700 1 -- 172.21.15.94:0/3586416283 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208004160 con 0x7f721810f040 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.767+0000 7f7215600700 1 -- 172.21.15.94:0/3586416283 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7208004300 con 0x7f721810f040 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.767+0000 7f7215600700 1 -- 172.21.15.94:0/3586416283 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208011af0 con 0x7f721810f040 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.767+0000 7f721d600700 1 -- 172.21.15.94:0/3586416283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 msgr2=0x7f721810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.767+0000 7f721d600700 1 --2- 172.21.15.94:0/3586416283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f721810f460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72080098a0 tx=0x7f7208009bb0 comp rx=0 tx=0).stop 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.768+0000 7f721d600700 1 -- 172.21.15.94:0/3586416283 shutdown_connections 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.768+0000 7f721d600700 1 --2- 172.21.15.94:0/3586416283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f721810f460 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.768+0000 7f721d600700 1 -- 172.21.15.94:0/3586416283 >> 172.21.15.94:0/3586416283 conn(0x7f721807ddf0 msgr2=0x7f7218080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.768+0000 7f721d600700 1 -- 172.21.15.94:0/3586416283 shutdown_connections 2024-04-25T14:50:49.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.768+0000 7f721d600700 1 -- 172.21.15.94:0/3586416283 wait complete. 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.769+0000 7f721d600700 1 Processor -- start 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.769+0000 7f721d600700 1 -- start start 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.769+0000 7f721d600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f72181a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.769+0000 7f721d600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7218112ff0 con 0x7f721810f040 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.770+0000 7f7216a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f72181a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.770+0000 7f7216a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f72181a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57358/0 (socket says 172.21.15.94:57358) 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.770+0000 7f7216a00700 1 -- 172.21.15.94:0/1567813228 learned_addr learned my addr 172.21.15.94:0/1567813228 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.770+0000 7f7216a00700 1 -- 172.21.15.94:0/1567813228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72181a2fa0 con 0x7f721810f040 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.770+0000 7f7216a00700 1 --2- 172.21.15.94:0/1567813228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f72181a2a60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7208011ac0 tx=0x7f7208011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.771+0000 7f720f400700 1 -- 172.21.15.94:0/1567813228 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7208003b20 con 0x7f721810f040 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.771+0000 7f720f400700 1 -- 172.21.15.94:0/1567813228 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f720801a8c0 con 0x7f721810f040 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.771+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72181a3230 con 0x7f721810f040 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.771+0000 7f720f400700 1 -- 172.21.15.94:0/1567813228 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72080248d0 con 0x7f721810f040 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.771+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72181a5f20 con 0x7f721810f040 2024-04-25T14:50:49.423 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.771+0000 7f720f400700 1 -- 172.21.15.94:0/1567813228 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7208022020 con 0x7f721810f040 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.771+0000 7f720f400700 1 -- 172.21.15.94:0/1567813228 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f720802dde0 con 0x7f721810f040 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.772+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7218113170 con 0x7f721810f040 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.775+0000 7f720f400700 1 -- 172.21.15.94:0/1567813228 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7208042e50 con 0x7f721810f040 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.852+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f72180640a0 con 0x7f721810f040 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.853+0000 7f720f400700 1 -- 172.21.15.94:0/1567813228 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f7208031020 con 0x7f721810f040 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.854+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 msgr2=0x7f72181a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.854+0000 7f721d600700 1 --2- 172.21.15.94:0/1567813228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f72181a2a60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7208011ac0 tx=0x7f7208011ba0 comp rx=0 tx=0).stop 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.854+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 shutdown_connections 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.854+0000 7f721d600700 1 --2- 172.21.15.94:0/1567813228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f721810f040 0x7f72181a2a60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.854+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 >> 172.21.15.94:0/1567813228 conn(0x7f721807ddf0 msgr2=0x7f721807f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.854+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 shutdown_connections 2024-04-25T14:50:49.424 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:50:48.854+0000 7f721d600700 1 -- 172.21.15.94:0/1567813228 wait complete. 2024-04-25T14:50:49.425 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-04-25T14:50:49.900 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:49 smithi094 systemd[1]: Stopping Ceph mon.a for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:50:50.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:49 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a[27488]: 2024-04-25T14:50:49.898+0000 7efdd9600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T14:50:50.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:49 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a[27488]: 2024-04-25T14:50:49.898+0000 7efdd9600700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T14:50:50.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:50 smithi094 podman[27737]: 2024-04-25 14:50:50.063996794 +0000 UTC m=+0.288415923 container died 71d4bcb43650e16597ac7c1c4c9620ca9fd10cd7cdb697f333f946b8dd233186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:50:50.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:50 smithi094 podman[27737]: 2024-04-25 14:50:50.509156705 +0000 UTC m=+0.733575853 container cleanup 71d4bcb43650e16597ac7c1c4c9620ca9fd10cd7cdb697f333f946b8dd233186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:50:50.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:50 smithi094 bash[27737]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a 2024-04-25T14:50:50.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:50 smithi094 podman[27749]: 2024-04-25 14:50:50.685169997 +0000 UTC m=+0.609534100 container remove 71d4bcb43650e16597ac7c1c4c9620ca9fd10cd7cdb697f333f946b8dd233186 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T14:50:51.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:51 smithi094 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T14:50:51.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:51 smithi094 systemd[1]: Stopped Ceph mon.a for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:50:51.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:51 smithi094 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.818s CPU time. 2024-04-25T14:50:51.395 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:51 smithi094 systemd[1]: Starting Ceph mon.a for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:50:51.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:51 smithi094 podman[27843]: 2024-04-25 14:50:51.424006035 +0000 UTC m=+0.037409022 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:50:51.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:51 smithi094 podman[27843]: 2024-04-25 14:50:51.519239382 +0000 UTC m=+0.132642406 container create f57cc4dcd35b57af7a01ab457ced21eec99dd2142f039a47d89cf2d8ea14093d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:50:56.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 podman[27843]: 2024-04-25 14:50:56.709223688 +0000 UTC m=+5.322626736 container init f57cc4dcd35b57af7a01ab457ced21eec99dd2142f039a47d89cf2d8ea14093d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T14:50:56.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 podman[27843]: 2024-04-25 14:50:56.718769493 +0000 UTC m=+5.332172482 container start f57cc4dcd35b57af7a01ab457ced21eec99dd2142f039a47d89cf2d8ea14093d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T14:50:56.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: pidfile_write: ignore empty --pid-file 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: load: jerasure load: lrc 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Git sha 0 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: DB SUMMARY 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: DB Session ID: 0SB87NXR4BXFBLFHIDIP 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T14:50:56.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104890 ; 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.env: 0x5614404b6cc0 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.info_log: 0x56145907fc40 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:50:56.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.statistics: (nil) 2024-04-25T14:50:56.966 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.use_fsync: 0 2024-04-25T14:50:56.966 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:50:56.966 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:50:56.966 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:50:56.966 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.db_log_dir: 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.wal_dir: 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:50:56.967 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.write_buffer_manager: 0x5614590130e0 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:50:56.968 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.unordered_write: 0 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.row_cache: None 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.wal_filter: None 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.wal_compression: 0 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:50:56.969 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:50:56.970 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_open_files: -1 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:50:56.971 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Compression algorithms supported: 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kZSTD supported: 0 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:50:56.972 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.merge_operator: 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_filter: None 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56145907fe00) 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_type: 0 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_index_type: 0 2024-04-25T14:50:56.973 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_shortening: 1 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: checksum: 4 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: no_block_cache: 0 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache: 0x5614590451f0 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_options: 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: capacity : 536870912 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_shard_bits : 4 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: strict_capacity_limit : 0 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_compressed: (nil) 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: persistent_cache: (nil) 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size: 4096 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size_deviation: 10 2024-04-25T14:50:56.974 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_restart_interval: 16 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_block_restart_interval: 1 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: metadata_block_size: 4096 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: partition_filters: 0 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: use_delta_encoding: 1 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: filter_policy: bloomfilter 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: whole_key_filtering: 1 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: verify_compression: 0 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: format_version: 5 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_index_compression: 1 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_align: 0 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: max_auto_readahead_size: 262144 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: prepopulate_block_cache: 0 2024-04-25T14:50:56.975 INFO:journalctl@ceph.mon.a.smithi094.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:50:56.976 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression: NoCompression 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.num_levels: 7 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:50:56.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:50:56.978 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:50:56.979 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:50:56.980 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:50:56.981 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.ttl: 2592000 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:50:56.982 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 76c2dc0f-1cef-484f-a0b2-f36bab79b6d4 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056656774233, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056656776204, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97920, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14131, "raw_average_key_size": 49, "raw_value_size": 90264, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714056656, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "76c2dc0f-1cef-484f-a0b2-f36bab79b6d4", "db_session_id": "0SB87NXR4BXFBLFHIDIP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056656776375, "job": 1, "event": "recovery_finished"} 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56145915a000 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: DB pointer 0x561459144000 2024-04-25T14:50:56.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** DB Stats ** 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: L0 2/0 99.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:50:56.984 INFO:journalctl@ceph.mon.a.smithi094.stdout: Sum 2/0 99.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 77.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:50:56.985 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative compaction: 0.00 GB write, 13.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval compaction: 0.00 GB write, 13.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache BinnedLRUCache@0x5614590451f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: starting mon.a rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???) e1 preinit fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???).mds e1 new map 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???).mds e1 print_map 2024-04-25T14:50:56.986 INFO:journalctl@ceph.mon.a.smithi094.stdout: e1 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout: legacy client fscid: -1 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout: No filesystems configured 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: monmap epoch 1 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: last_changed 2024-04-25T14:50:41.313017+0000 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:50:56.987 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: min_mon_release 19 (squid) 2024-04-25T14:50:56.988 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: election_strategy: 1 2024-04-25T14:50:56.988 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:50:56.988 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: fsmap 2024-04-25T14:50:56.988 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T14:50:56.988 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:56 smithi094 ceph-mon[27862]: mgrmap e1: no daemons active 2024-04-25T14:50:59.668 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T14:50:59.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:59 smithi094 bash[27843]: f57cc4dcd35b57af7a01ab457ced21eec99dd2142f039a47d89cf2d8ea14093d 2024-04-25T14:50:59.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:50:59 smithi094 systemd[1]: Started Ceph mon.a for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.662+0000 7f9f6a600700 1 Processor -- start 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.662+0000 7f9f6a600700 1 -- start start 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.662+0000 7f9f6a600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6410aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.662+0000 7f9f6a600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6410b010 con 0x7f9f6410a620 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.663+0000 7f9f69200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6410aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.663+0000 7f9f69200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6410aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58530/0 (socket says 172.21.15.94:58530) 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.663+0000 7f9f69200700 1 -- 172.21.15.94:0/4204397459 learned_addr learned my addr 172.21.15.94:0/4204397459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.664+0000 7f9f69200700 1 -- 172.21.15.94:0/4204397459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6410b7d0 con 0x7f9f6410a620 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.665+0000 7f9f69200700 1 --2- 172.21.15.94:0/4204397459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6410aa40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f580098a0 tx=0x7f9f58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25992a54bef023ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:02.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.665+0000 7f9f63400700 1 -- 172.21.15.94:0/4204397459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f58004160 con 0x7f9f6410a620 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.665+0000 7f9f63400700 1 -- 172.21.15.94:0/4204397459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f58004300 con 0x7f9f6410a620 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.665+0000 7f9f63400700 1 -- 172.21.15.94:0/4204397459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f58011af0 con 0x7f9f6410a620 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.666+0000 7f9f6a600700 1 -- 172.21.15.94:0/4204397459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 msgr2=0x7f9f6410aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.666+0000 7f9f6a600700 1 --2- 172.21.15.94:0/4204397459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6410aa40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9f580098a0 tx=0x7f9f58009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.666+0000 7f9f6a600700 1 -- 172.21.15.94:0/4204397459 shutdown_connections 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.666+0000 7f9f6a600700 1 --2- 172.21.15.94:0/4204397459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6410aa40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.666+0000 7f9f6a600700 1 -- 172.21.15.94:0/4204397459 >> 172.21.15.94:0/4204397459 conn(0x7f9f6407dbf0 msgr2=0x7f9f641084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.666+0000 7f9f6a600700 1 -- 172.21.15.94:0/4204397459 shutdown_connections 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.666+0000 7f9f6a600700 1 -- 172.21.15.94:0/4204397459 wait complete. 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.667+0000 7f9f6a600700 1 Processor -- start 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.667+0000 7f9f6a600700 1 -- start start 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.668+0000 7f9f6a600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6419e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.668+0000 7f9f6a600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6410ea00 con 0x7f9f6410a620 2024-04-25T14:51:02.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.668+0000 7f9f69200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6419e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.668+0000 7f9f69200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6419e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58536/0 (socket says 172.21.15.94:58536) 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.668+0000 7f9f69200700 1 -- 172.21.15.94:0/2956780119 learned_addr learned my addr 172.21.15.94:0/2956780119 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.668+0000 7f9f69200700 1 -- 172.21.15.94:0/2956780119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6419ea20 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.669+0000 7f9f69200700 1 --2- 172.21.15.94:0/2956780119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6419e4e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f58003f40 tx=0x7f9f58011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.669+0000 7f9f61600700 1 -- 172.21.15.94:0/2956780119 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f580038b0 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.669+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6419ecb0 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.669+0000 7f9f61600700 1 -- 172.21.15.94:0/2956780119 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9f58003a50 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.669+0000 7f9f61600700 1 -- 172.21.15.94:0/2956780119 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f5801a5c0 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.669+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f641a19a0 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.670+0000 7f9f61600700 1 -- 172.21.15.94:0/2956780119 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9f5801ac00 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.670+0000 7f9f61600700 1 -- 172.21.15.94:0/2956780119 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9f58019700 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.670+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f6410eb80 con 0x7f9f6410a620 2024-04-25T14:51:02.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.674+0000 7f9f61600700 1 -- 172.21.15.94:0/2956780119 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9f58019900 con 0x7f9f6410a620 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.749+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f9f641a1b70 con 0x7f9f6410a620 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.753+0000 7f9f61600700 1 -- 172.21.15.94:0/2956780119 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f9f58019ae0 con 0x7f9f6410a620 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.755+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 msgr2=0x7f9f6419e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.755+0000 7f9f6a600700 1 --2- 172.21.15.94:0/2956780119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6419e4e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9f58003f40 tx=0x7f9f58011890 comp rx=0 tx=0).stop 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.755+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 shutdown_connections 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.755+0000 7f9f6a600700 1 --2- 172.21.15.94:0/2956780119 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f6410a620 0x7f9f6419e4e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.755+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 >> 172.21.15.94:0/2956780119 conn(0x7f9f6407dbf0 msgr2=0x7f9f64107c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.755+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 shutdown_connections 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:01.755+0000 7f9f6a600700 1 -- 172.21.15.94:0/2956780119 wait complete. 2024-04-25T14:51:02.438 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T14:51:02.463 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:51:02.463 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-04-25T14:51:02.464 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T14:51:02.466 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T14:51:02.874 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.x 2024-04-25T14:51:02.874 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T14:51:03.136 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f04d929c-0312-11ef-bc93-c7b262605968.target.wants/ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f04d929c-0312-11ef-bc93-c7b262605968@.service. 2024-04-25T14:51:03.151 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:02 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2956780119' entity='client.admin' 2024-04-25T14:51:03.461 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:03 smithi094 systemd[1]: Starting Ceph mgr.x for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:51:03.828 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:03 smithi094 podman[28076]: 2024-04-25 14:51:03.445706475 +0000 UTC m=+0.040069464 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T14:51:03.828 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:03 smithi094 podman[28076]: 2024-04-25 14:51:03.565691053 +0000 UTC m=+0.160054031 container create 2441008644f040a07eb7176ba01e877a9cd4dbd07c95fc46788dfa00d6ec6388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T14:51:03.828 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:03 smithi094 podman[28076]: 2024-04-25 14:51:03.826800928 +0000 UTC m=+0.421163884 container init 2441008644f040a07eb7176ba01e877a9cd4dbd07c95fc46788dfa00d6ec6388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:51:03.965 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:51:03.965 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T14:51:03.982 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:51:03.982 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T14:51:03.982 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T14:51:03.983 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T14:51:04.001 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:51:04.001 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T14:51:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:51:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T14:51:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T14:51:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T14:51:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-04-25T14:51:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-04-25T14:51:04.083 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:03 smithi094 podman[28076]: 2024-04-25 14:51:03.835071803 +0000 UTC m=+0.429434784 container start 2441008644f040a07eb7176ba01e877a9cd4dbd07c95fc46788dfa00d6ec6388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:51:04.083 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:03 smithi094 bash[28076]: 2441008644f040a07eb7176ba01e877a9cd4dbd07c95fc46788dfa00d6ec6388 2024-04-25T14:51:04.083 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:03 smithi094 systemd[1]: Started Ceph mgr.x for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:51:04.354 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:04.081+0000 7fcc34d98200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:51:04.354 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:04.329+0000 7fcc34d98200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:51:04.606 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:04.465+0000 7fcc34d98200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:51:04.885 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:04 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2367522712' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:51:05.211 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:04.883+0000 7fcc34d98200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:51:05.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:04.997+0000 7fcc34d98200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:51:05.403 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:51:05.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:51:05.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f04d929c-0312-11ef-bc93-c7b262605968", 2024-04-25T14:51:05.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:51:05.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:51:05.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:51:05.404 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:51:05.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:05.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:51:05.405 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:51:05.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:51:05.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:05.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:51:05.406 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:05.407 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:51:05.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:05.408 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:51:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:51:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:51:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:51:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:50:45.400907+0000", 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:51:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:51:05.418 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.604+0000 7fcc27200700 1 Processor -- start 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.605+0000 7fcc27200700 1 -- start start 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.605+0000 7fcc27200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2007b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.605+0000 7fcc27200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2007b760 con 0x7fcc2007cd30 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.606+0000 7fcc1fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.606+0000 7fcc1fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2007b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58562/0 (socket says 172.21.15.94:58562) 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.606+0000 7fcc1fe00700 1 -- 172.21.15.94:0/1827426052 learned_addr learned my addr 172.21.15.94:0/1827426052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.606+0000 7fcc1fe00700 1 -- 172.21.15.94:0/1827426052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2007b8e0 con 0x7fcc2007cd30 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.607+0000 7fcc1fe00700 1 --2- 172.21.15.94:0/1827426052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2007b190 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e742fea421d475d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.607+0000 7fcc1ea00700 1 -- 172.21.15.94:0/1827426052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10004160 con 0x7fcc2007cd30 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.607+0000 7fcc1ea00700 1 -- 172.21.15.94:0/1827426052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc10004300 con 0x7fcc2007cd30 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.607+0000 7fcc1ea00700 1 -- 172.21.15.94:0/1827426052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10011af0 con 0x7fcc2007cd30 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.608+0000 7fcc27200700 1 -- 172.21.15.94:0/1827426052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 msgr2=0x7fcc2007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.608+0000 7fcc27200700 1 --2- 172.21.15.94:0/1827426052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2007b190 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098a0 tx=0x7fcc10009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:05.419 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.608+0000 7fcc27200700 1 -- 172.21.15.94:0/1827426052 shutdown_connections 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.608+0000 7fcc27200700 1 --2- 172.21.15.94:0/1827426052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2007b190 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.608+0000 7fcc27200700 1 -- 172.21.15.94:0/1827426052 >> 172.21.15.94:0/1827426052 conn(0x7fcc20103bf0 msgr2=0x7fcc20106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.608+0000 7fcc27200700 1 -- 172.21.15.94:0/1827426052 shutdown_connections 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.608+0000 7fcc27200700 1 -- 172.21.15.94:0/1827426052 wait complete. 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.609+0000 7fcc27200700 1 Processor -- start 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.610+0000 7fcc27200700 1 -- start start 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.610+0000 7fcc27200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.610+0000 7fcc27200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2010eb00 con 0x7fcc2007cd30 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.610+0000 7fcc1fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.610+0000 7fcc1fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58578/0 (socket says 172.21.15.94:58578) 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.610+0000 7fcc1fe00700 1 -- 172.21.15.94:0/2367522712 learned_addr learned my addr 172.21.15.94:0/2367522712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.611+0000 7fcc1fe00700 1 -- 172.21.15.94:0/2367522712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2019ebf0 con 0x7fcc2007cd30 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.611+0000 7fcc1fe00700 1 --2- 172.21.15.94:0/2367522712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2019e6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc10011ac0 tx=0x7fcc10011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.611+0000 7fcc1cc00700 1 -- 172.21.15.94:0/2367522712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc10003b20 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.611+0000 7fcc1cc00700 1 -- 172.21.15.94:0/2367522712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcc1001a8c0 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.611+0000 7fcc1cc00700 1 -- 172.21.15.94:0/2367522712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc100248d0 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.611+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2019ee80 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.611+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201a1b20 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.612+0000 7fcc1cc00700 1 -- 172.21.15.94:0/2367522712 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcc10022020 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.612+0000 7fcc1cc00700 1 -- 172.21.15.94:0/2367522712 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcc100193c0 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.613+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc200640a0 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.617+0000 7fcc1cc00700 1 -- 172.21.15.94:0/2367522712 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcc1002ddf0 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.696+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fcc201a1fc0 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.697+0000 7fcc1cc00700 1 -- 172.21.15.94:0/2367522712 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fcc1001aa70 con 0x7fcc2007cd30 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.698+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 msgr2=0x7fcc2019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.698+0000 7fcc27200700 1 --2- 172.21.15.94:0/2367522712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2019e6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcc10011ac0 tx=0x7fcc10011ba0 comp rx=0 tx=0).stop 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.698+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 shutdown_connections 2024-04-25T14:51:05.421 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.698+0000 7fcc27200700 1 --2- 172.21.15.94:0/2367522712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc2007cd30 0x7fcc2019e6b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:05.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.698+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 >> 172.21.15.94:0/2367522712 conn(0x7fcc20103bf0 msgr2=0x7fcc20104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:05.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.699+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 shutdown_connections 2024-04-25T14:51:05.422 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:04.699+0000 7fcc27200700 1 -- 172.21.15.94:0/2367522712 wait complete. 2024-04-25T14:51:05.422 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-04-25T14:51:06.961 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:06.552+0000 7fcc34d98200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:51:06.962 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:06.772+0000 7fcc34d98200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:51:07.361 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:06.982+0000 7fcc34d98200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:51:07.711 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:07.360+0000 7fcc34d98200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:51:07.711 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:07.489+0000 7fcc34d98200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:51:07.961 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:07.750+0000 7fcc34d98200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:51:08.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:08 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3682053147' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:51:08.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:08 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:08.006+0000 7fcc34d98200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:51:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:51:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:51:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f04d929c-0312-11ef-bc93-c7b262605968", 2024-04-25T14:51:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:51:08.886 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:51:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:51:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:51:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:51:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:51:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:51:08.888 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:51:08.889 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:51:08.896 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:51:08.897 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:50:45.400907+0000", 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:51:08.898 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.050+0000 7f9b8f600700 1 Processor -- start 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.050+0000 7f9b8f600700 1 -- start start 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.051+0000 7f9b8f600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b8807b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.051+0000 7f9b8f600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8807b7e0 con 0x7f9b8807cdb0 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.051+0000 7f9b8cc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b8807b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.051+0000 7f9b8cc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b8807b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59084/0 (socket says 172.21.15.94:59084) 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.052+0000 7f9b8cc00700 1 -- 172.21.15.94:0/160575292 learned_addr learned my addr 172.21.15.94:0/160575292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.052+0000 7f9b8cc00700 1 -- 172.21.15.94:0/160575292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8807b960 con 0x7f9b8807cdb0 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.052+0000 7f9b8cc00700 1 --2- 172.21.15.94:0/160575292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b8807b210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0098a0 tx=0x7f9b7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9385e998ab40f272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.053+0000 7f9b87400700 1 -- 172.21.15.94:0/160575292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7c004160 con 0x7f9b8807cdb0 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.053+0000 7f9b87400700 1 -- 172.21.15.94:0/160575292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b7c004300 con 0x7f9b8807cdb0 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.053+0000 7f9b87400700 1 -- 172.21.15.94:0/160575292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7c011af0 con 0x7f9b8807cdb0 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.053+0000 7f9b8f600700 1 -- 172.21.15.94:0/160575292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 msgr2=0x7f9b8807b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.053+0000 7f9b8f600700 1 --2- 172.21.15.94:0/160575292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b8807b210 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0098a0 tx=0x7f9b7c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:08.899 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.054+0000 7f9b8f600700 1 -- 172.21.15.94:0/160575292 shutdown_connections 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.054+0000 7f9b8f600700 1 --2- 172.21.15.94:0/160575292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b8807b210 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.054+0000 7f9b8f600700 1 -- 172.21.15.94:0/160575292 >> 172.21.15.94:0/160575292 conn(0x7f9b88103be0 msgr2=0x7f9b88106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.054+0000 7f9b8f600700 1 -- 172.21.15.94:0/160575292 shutdown_connections 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.054+0000 7f9b8f600700 1 -- 172.21.15.94:0/160575292 wait complete. 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.055+0000 7f9b8f600700 1 Processor -- start 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.056+0000 7f9b8f600700 1 -- start start 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.056+0000 7f9b8f600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b881a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.056+0000 7f9b8f600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8810a710 con 0x7f9b8807cdb0 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.056+0000 7f9b8cc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b881a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.056+0000 7f9b8cc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b881a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59088/0 (socket says 172.21.15.94:59088) 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.056+0000 7f9b8cc00700 1 -- 172.21.15.94:0/3682053147 learned_addr learned my addr 172.21.15.94:0/3682053147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.057+0000 7f9b8cc00700 1 -- 172.21.15.94:0/3682053147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b881a3010 con 0x7f9b8807cdb0 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.057+0000 7f9b8cc00700 1 --2- 172.21.15.94:0/3682053147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b881a2ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.057+0000 7f9b85600700 1 -- 172.21.15.94:0/3682053147 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7c003b20 con 0x7f9b8807cdb0 2024-04-25T14:51:08.900 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.057+0000 7f9b85600700 1 -- 172.21.15.94:0/3682053147 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9b7c003cc0 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.057+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b881a32a0 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.057+0000 7f9b85600700 1 -- 172.21.15.94:0/3682053147 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7c024800 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.057+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b881a5f90 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.058+0000 7f9b85600700 1 -- 172.21.15.94:0/3682053147 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b7c022020 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.058+0000 7f9b85600700 1 -- 172.21.15.94:0/3682053147 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9b7c019d60 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.058+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8819c060 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.061+0000 7f9b85600700 1 -- 172.21.15.94:0/3682053147 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9b7c0042a0 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.141+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9b8802e140 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.142+0000 7f9b85600700 1 -- 172.21.15.94:0/3682053147 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f9b7c024d30 con 0x7f9b8807cdb0 2024-04-25T14:51:08.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.144+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 msgr2=0x7f9b881a2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.144+0000 7f9b8f600700 1 --2- 172.21.15.94:0/3682053147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b881a2ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c011b70 comp rx=0 tx=0).stop 2024-04-25T14:51:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.144+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 shutdown_connections 2024-04-25T14:51:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.144+0000 7f9b8f600700 1 --2- 172.21.15.94:0/3682053147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b8807cdb0 0x7f9b881a2ad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:08.902 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.144+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 >> 172.21.15.94:0/3682053147 conn(0x7f9b88103be0 msgr2=0x7f9b88105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.144+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 shutdown_connections 2024-04-25T14:51:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:08.144+0000 7f9b8f600700 1 -- 172.21.15.94:0/3682053147 wait complete. 2024-04-25T14:51:08.903 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-04-25T14:51:09.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:08 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:08.828+0000 7fcc34d98200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:51:09.711 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:09.377+0000 7fcc34d98200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:51:09.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:09.502+0000 7fcc34d98200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:51:10.211 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:09.762+0000 7fcc34d98200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:51:10.711 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:10 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:10.400+0000 7fcc34d98200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:51:10.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:10 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:10.521+0000 7fcc34d98200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:51:11.146 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:10 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:10.837+0000 7fcc34d98200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:51:11.146 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:11.017+0000 7fcc34d98200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:51:11.146 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:11.144+0000 7fcc34d98200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:51:11.441 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:11.298+0000 7fcc34d98200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:51:11.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:11 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/40263311' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:51:11.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:11.681+0000 7fcc34d98200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:51:12.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f04d929c-0312-11ef-bc93-c7b262605968", 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:51:12.232 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:51:12.233 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:51:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:51:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:51:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:51:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:51:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:51:12.242 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:51:12.243 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:50:45.400907+0000", 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:51:12.244 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.440+0000 7f5fe3e00700 1 Processor -- start 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.440+0000 7f5fe3e00700 1 -- start start 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.440+0000 7f5fe3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc10b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.440+0000 7f5fe3e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc063d20 con 0x7f5fdc109080 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.441+0000 7f5fe1400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc10b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.441+0000 7f5fe1400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc10b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59098/0 (socket says 172.21.15.94:59098) 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.441+0000 7f5fe1400700 1 -- 172.21.15.94:0/3942153464 learned_addr learned my addr 172.21.15.94:0/3942153464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.441+0000 7f5fe1400700 1 -- 172.21.15.94:0/3942153464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc10b9b0 con 0x7f5fdc109080 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.442+0000 7f5fe1400700 1 --2- 172.21.15.94:0/3942153464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc10b470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00098a0 tx=0x7f5fd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a9759f122f2063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.442+0000 7f5fdbe00700 1 -- 172.21.15.94:0/3942153464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd0004160 con 0x7f5fdc109080 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.442+0000 7f5fdbe00700 1 -- 172.21.15.94:0/3942153464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fd0004300 con 0x7f5fdc109080 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.442+0000 7f5fdbe00700 1 -- 172.21.15.94:0/3942153464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd0011af0 con 0x7f5fdc109080 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.443+0000 7f5fe3e00700 1 -- 172.21.15.94:0/3942153464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 msgr2=0x7f5fdc10b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:12.245 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.443+0000 7f5fe3e00700 1 --2- 172.21.15.94:0/3942153464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc10b470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00098a0 tx=0x7f5fd0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.443+0000 7f5fe3e00700 1 -- 172.21.15.94:0/3942153464 shutdown_connections 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.443+0000 7f5fe3e00700 1 --2- 172.21.15.94:0/3942153464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc10b470 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.443+0000 7f5fe3e00700 1 -- 172.21.15.94:0/3942153464 >> 172.21.15.94:0/3942153464 conn(0x7f5fdc102ab0 msgr2=0x7f5fdc104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.443+0000 7f5fe3e00700 1 -- 172.21.15.94:0/3942153464 shutdown_connections 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.443+0000 7f5fe3e00700 1 -- 172.21.15.94:0/3942153464 wait complete. 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.444+0000 7f5fe3e00700 1 Processor -- start 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.445+0000 7f5fe3e00700 1 -- start start 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.445+0000 7f5fe3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc108900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.445+0000 7f5fe3e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10c900 con 0x7f5fdc109080 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.445+0000 7f5fe1400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.445+0000 7f5fe1400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59106/0 (socket says 172.21.15.94:59106) 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.445+0000 7f5fe1400700 1 -- 172.21.15.94:0/40263311 learned_addr learned my addr 172.21.15.94:0/40263311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.446+0000 7f5fe1400700 1 -- 172.21.15.94:0/40263311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc108e40 con 0x7f5fdc109080 2024-04-25T14:51:12.246 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.446+0000 7f5fe1400700 1 --2- 172.21.15.94:0/40263311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc108900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0011ac0 tx=0x7f5fd0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.446+0000 7f5fda000700 1 -- 172.21.15.94:0/40263311 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd0003b20 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.446+0000 7f5fda000700 1 -- 172.21.15.94:0/40263311 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fd001a8c0 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.447+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc107030 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.447+0000 7f5fda000700 1 -- 172.21.15.94:0/40263311 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fd00248d0 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.447+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc107570 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.447+0000 7f5fda000700 1 -- 172.21.15.94:0/40263311 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5fd0022020 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.447+0000 7f5fda000700 1 -- 172.21.15.94:0/40263311 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5fd002dde0 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.448+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc0640a0 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.450+0000 7f5fda000700 1 -- 172.21.15.94:0/40263311 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5fd0042e50 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.530+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5fdc107ba0 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.531+0000 7f5fda000700 1 -- 172.21.15.94:0/40263311 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f5fd0031020 con 0x7f5fdc109080 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.532+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 msgr2=0x7f5fdc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:12.247 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.532+0000 7f5fe3e00700 1 --2- 172.21.15.94:0/40263311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc108900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0011ac0 tx=0x7f5fd0011ba0 comp rx=0 tx=0).stop 2024-04-25T14:51:12.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.532+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 shutdown_connections 2024-04-25T14:51:12.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.532+0000 7f5fe3e00700 1 --2- 172.21.15.94:0/40263311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fdc109080 0x7f5fdc108900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:12.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.532+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 >> 172.21.15.94:0/40263311 conn(0x7f5fdc102ab0 msgr2=0x7f5fdc103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:12.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.532+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 shutdown_connections 2024-04-25T14:51:12.248 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:11.532+0000 7f5fe3e00700 1 -- 172.21.15.94:0/40263311 wait complete. 2024-04-25T14:51:12.248 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (3/15)... 2024-04-25T14:51:12.461 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:12.446+0000 7fcc34d98200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:51:13.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:12.726+0000 7fcc34d98200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:51:13.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: Activating manager daemon x 2024-04-25T14:51:13.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: mgrmap e2: x(active, starting, since 0.0108035s) 2024-04-25T14:51:13.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:51:13.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:51:13.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:51:13.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:51:13.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:51:13.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: Manager daemon x is now available 2024-04-25T14:51:13.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:51:13.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:51:13.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' 2024-04-25T14:51:13.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' 2024-04-25T14:51:13.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:12 smithi094 ceph-mon[27862]: from='mgr.14100 172.21.15.94:0/293156732' entity='mgr.x' 2024-04-25T14:51:15.194 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:14 smithi094 ceph-mon[27862]: mgrmap e3: x(active, since 1.01837s) 2024-04-25T14:51:15.886 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:15 smithi094 ceph-mon[27862]: mgrmap e4: x(active, since 2s) 2024-04-25T14:51:15.886 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:15 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/79031560' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T14:51:15.901 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:51:15.905 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:51:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "f04d929c-0312-11ef-bc93-c7b262605968", 2024-04-25T14:51:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T14:51:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T14:51:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T14:51:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T14:51:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T14:51:15.916 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T14:51:15.916 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:15.917 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T14:51:15.918 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T14:51:15.919 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T14:50:45.400907+0000", 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.876+0000 7f3903600700 1 Processor -- start 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.876+0000 7f3903600700 1 -- start start 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.877+0000 7f3903600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc1089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.877+0000 7f3903600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc108fb0 con 0x7f38fc1085c0 2024-04-25T14:51:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.877+0000 7f3900c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc1089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.877+0000 7f3900c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc1089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59144/0 (socket says 172.21.15.94:59144) 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.877+0000 7f3900c00700 1 -- 172.21.15.94:0/3019069357 learned_addr learned my addr 172.21.15.94:0/3019069357 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.878+0000 7f3900c00700 1 -- 172.21.15.94:0/3019069357 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc1097c0 con 0x7f38fc1085c0 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.878+0000 7f3900c00700 1 --2- 172.21.15.94:0/3019069357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc1089e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e80 tx=0x7f38f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585d8145642f4fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.878+0000 7f38fb400700 1 -- 172.21.15.94:0/3019069357 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f0017070 con 0x7f38fc1085c0 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.878+0000 7f38fb400700 1 -- 172.21.15.94:0/3019069357 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f38f000b8c0 con 0x7f38fc1085c0 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.878+0000 7f38fb400700 1 -- 172.21.15.94:0/3019069357 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f0004490 con 0x7f38fc1085c0 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.879+0000 7f3903600700 1 -- 172.21.15.94:0/3019069357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 msgr2=0x7f38fc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.879+0000 7f3903600700 1 --2- 172.21.15.94:0/3019069357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc1089e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009e80 tx=0x7f38f000b330 comp rx=0 tx=0).stop 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.879+0000 7f3903600700 1 -- 172.21.15.94:0/3019069357 shutdown_connections 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.879+0000 7f3903600700 1 --2- 172.21.15.94:0/3019069357 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc1089e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.879+0000 7f3903600700 1 -- 172.21.15.94:0/3019069357 >> 172.21.15.94:0/3019069357 conn(0x7f38fc103be0 msgr2=0x7f38fc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.879+0000 7f3903600700 1 -- 172.21.15.94:0/3019069357 shutdown_connections 2024-04-25T14:51:15.921 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.879+0000 7f3903600700 1 -- 172.21.15.94:0/3019069357 wait complete. 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.881+0000 7f3903600700 1 Processor -- start 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.881+0000 7f3903600700 1 -- start start 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.881+0000 7f3903600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.881+0000 7f3903600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc10c9f0 con 0x7f38fc1085c0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.881+0000 7f3900c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.881+0000 7f3900c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59148/0 (socket says 172.21.15.94:59148) 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.882+0000 7f3900c00700 1 -- 172.21.15.94:0/79031560 learned_addr learned my addr 172.21.15.94:0/79031560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.882+0000 7f3900c00700 1 -- 172.21.15.94:0/79031560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc19ebd0 con 0x7f38fc1085c0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.882+0000 7f3900c00700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc19e690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38f0004a60 tx=0x7f38f000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.882+0000 7f38f9600700 1 -- 172.21.15.94:0/79031560 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f0020020 con 0x7f38fc1085c0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.882+0000 7f38f9600700 1 -- 172.21.15.94:0/79031560 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f38f0003c60 con 0x7f38fc1085c0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.882+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc19ee60 con 0x7f38fc1085c0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.882+0000 7f38f9600700 1 -- 172.21.15.94:0/79031560 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38f0017040 con 0x7f38fc1085c0 2024-04-25T14:51:15.922 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.883+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a1b50 con 0x7f38fc1085c0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.884+0000 7f38f9600700 1 -- 172.21.15.94:0/79031560 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f38f0023020 con 0x7f38fc1085c0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.884+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc10cb70 con 0x7f38fc1085c0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.884+0000 7f38f9600700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f38e4039560 0x7f38e403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.884+0000 7f38f9600700 1 -- 172.21.15.94:0/79031560 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f38f000f050 con 0x7f38fc1085c0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.885+0000 7f38fbe00700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f38e4039560 0x7f38e403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.885+0000 7f38fbe00700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f38e4039560 0x7f38e403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006fd0 tx=0x7f38e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:14.891+0000 7f38f9600700 1 -- 172.21.15.94:0/79031560 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38f002a020 con 0x7f38fc1085c0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.192+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f38fc1a1d20 con 0x7f38fc1085c0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.193+0000 7f38f9600700 1 -- 172.21.15.94:0/79031560 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f38f002c300 con 0x7f38fc1085c0 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f38e4039560 msgr2=0x7f38e403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f38e4039560 0x7f38e403ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38e8006fd0 tx=0x7f38e8006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 msgr2=0x7f38fc19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc19e690 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f38f0004a60 tx=0x7f38f000bd50 comp rx=0 tx=0).stop 2024-04-25T14:51:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 shutdown_connections 2024-04-25T14:51:15.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f38e4039560 0x7f38e403ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:15.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 --2- 172.21.15.94:0/79031560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38fc1085c0 0x7f38fc19e690 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:15.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 >> 172.21.15.94:0/79031560 conn(0x7f38fc103be0 msgr2=0x7f38fc104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:15.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 shutdown_connections 2024-04-25T14:51:15.924 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:15.197+0000 7f3903600700 1 -- 172.21.15.94:0/79031560 wait complete. 2024-04-25T14:51:15.924 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-04-25T14:51:17.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:16 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1143754674' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T14:51:17.379 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.476+0000 7f1bc1200700 1 Processor -- start 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.476+0000 7f1bc1200700 1 -- start start 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.476+0000 7f1bc1200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc10f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.476+0000 7f1bc1200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc10fa30 con 0x7f1bbc10f040 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.477+0000 7f1bb9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.477+0000 7f1bb9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc10f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59154/0 (socket says 172.21.15.94:59154) 2024-04-25T14:51:17.380 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.477+0000 7f1bb9e00700 1 -- 172.21.15.94:0/4580042 learned_addr learned my addr 172.21.15.94:0/4580042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.477+0000 7f1bb9e00700 1 -- 172.21.15.94:0/4580042 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc10fdc0 con 0x7f1bbc10f040 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.477+0000 7f1bb9e00700 1 --2- 172.21.15.94:0/4580042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc10f460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009e80 tx=0x7f1bac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c5e7b98ec9953e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.478+0000 7f1bb9400700 1 -- 172.21.15.94:0/4580042 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bac017070 con 0x7f1bbc10f040 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.478+0000 7f1bb9400700 1 -- 172.21.15.94:0/4580042 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1bac00b8c0 con 0x7f1bbc10f040 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.478+0000 7f1bb9400700 1 -- 172.21.15.94:0/4580042 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bac004490 con 0x7f1bbc10f040 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.478+0000 7f1bc1200700 1 -- 172.21.15.94:0/4580042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 msgr2=0x7f1bbc10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.478+0000 7f1bc1200700 1 --2- 172.21.15.94:0/4580042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc10f460 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009e80 tx=0x7f1bac00b330 comp rx=0 tx=0).stop 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.479+0000 7f1bc1200700 1 -- 172.21.15.94:0/4580042 shutdown_connections 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.479+0000 7f1bc1200700 1 --2- 172.21.15.94:0/4580042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc10f460 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.479+0000 7f1bc1200700 1 -- 172.21.15.94:0/4580042 >> 172.21.15.94:0/4580042 conn(0x7f1bbc07ddf0 msgr2=0x7f1bbc080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.479+0000 7f1bc1200700 1 -- 172.21.15.94:0/4580042 shutdown_connections 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.479+0000 7f1bc1200700 1 -- 172.21.15.94:0/4580042 wait complete. 2024-04-25T14:51:17.381 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.480+0000 7f1bc1200700 1 Processor -- start 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.480+0000 7f1bc1200700 1 -- start start 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.480+0000 7f1bc1200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc1a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.480+0000 7f1bc1200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1bbc112ff0 con 0x7f1bbc10f040 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.481+0000 7f1bb9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc1a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.481+0000 7f1bb9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc1a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59166/0 (socket says 172.21.15.94:59166) 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.481+0000 7f1bb9e00700 1 -- 172.21.15.94:0/1143754674 learned_addr learned my addr 172.21.15.94:0/1143754674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.481+0000 7f1bb9e00700 1 -- 172.21.15.94:0/1143754674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bbc1a2f40 con 0x7f1bbc10f040 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.481+0000 7f1bb9e00700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc1a2a00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b7b0 tx=0x7f1bac00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.482+0000 7f1bb3400700 1 -- 172.21.15.94:0/1143754674 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bac020020 con 0x7f1bbc10f040 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.482+0000 7f1bb3400700 1 -- 172.21.15.94:0/1143754674 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1bac003d10 con 0x7f1bbc10f040 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.482+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bbc1a31d0 con 0x7f1bbc10f040 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.482+0000 7f1bb3400700 1 -- 172.21.15.94:0/1143754674 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1bac017040 con 0x7f1bbc10f040 2024-04-25T14:51:17.382 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.482+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bbc1a5ec0 con 0x7f1bbc10f040 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.483+0000 7f1bb3400700 1 -- 172.21.15.94:0/1143754674 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f1bac023020 con 0x7f1bbc10f040 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.483+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bbc113170 con 0x7f1bbc10f040 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.483+0000 7f1bb3400700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f1ba0039560 0x7f1ba003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.483+0000 7f1bb3400700 1 -- 172.21.15.94:0/1143754674 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1bac00f050 con 0x7f1bbc10f040 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.484+0000 7f1bb1400700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f1ba0039560 0x7f1ba003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.485+0000 7f1bb1400700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f1ba0039560 0x7f1ba003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006fd0 tx=0x7f1ba4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.489+0000 7f1bb3400700 1 -- 172.21.15.94:0/1143754674 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bac02a020 con 0x7f1bbc10f040 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.712+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1bbc0640a0 con 0x7f1bbc10f040 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.714+0000 7f1bb3400700 1 -- 172.21.15.94:0/1143754674 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f1bac02c300 con 0x7f1bbc10f040 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f1ba0039560 msgr2=0x7f1ba003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f1ba0039560 0x7f1ba003ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4006fd0 tx=0x7f1ba4006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 msgr2=0x7f1bbc1a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc1a2a00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b7b0 tx=0x7f1bac00be90 comp rx=0 tx=0).stop 2024-04-25T14:51:17.383 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 shutdown_connections 2024-04-25T14:51:17.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f1ba0039560 0x7f1ba003ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:17.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 --2- 172.21.15.94:0/1143754674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1bbc10f040 0x7f1bbc1a2a00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:17.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 >> 172.21.15.94:0/1143754674 conn(0x7f1bbc07ddf0 msgr2=0x7f1bbc07f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:17.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 shutdown_connections 2024-04-25T14:51:17.384 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:16.717+0000 7f1bc1200700 1 -- 172.21.15.94:0/1143754674 wait complete. 2024-04-25T14:51:17.384 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-04-25T14:51:18.623 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:18 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3190734523' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T14:51:18.624 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:18.624 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:18.961 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:18.621+0000 7f5e4f323200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:51:18.962 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:18.864+0000 7f5e4f323200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:51:19.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.983+0000 7fd238a00700 1 Processor -- start 2024-04-25T14:51:19.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.983+0000 7fd238a00700 1 -- start start 2024-04-25T14:51:19.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.984+0000 7fd238a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23407b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:19.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.984+0000 7fd238a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd23407b760 con 0x7fd23407cd30 2024-04-25T14:51:19.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.984+0000 7fd231e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23407b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:19.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.984+0000 7fd231e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23407b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60780/0 (socket says 172.21.15.94:60780) 2024-04-25T14:51:19.018 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.984+0000 7fd231e00700 1 -- 172.21.15.94:0/4250647062 learned_addr learned my addr 172.21.15.94:0/4250647062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.985+0000 7fd231e00700 1 -- 172.21.15.94:0/4250647062 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23407b8e0 con 0x7fd23407cd30 2024-04-25T14:51:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.985+0000 7fd231e00700 1 --2- 172.21.15.94:0/4250647062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23407b190 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2240098a0 tx=0x7fd224009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367b110c394d0e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.985+0000 7fd230a00700 1 -- 172.21.15.94:0/4250647062 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd224004160 con 0x7fd23407cd30 2024-04-25T14:51:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.985+0000 7fd230a00700 1 -- 172.21.15.94:0/4250647062 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd224004300 con 0x7fd23407cd30 2024-04-25T14:51:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.985+0000 7fd230a00700 1 -- 172.21.15.94:0/4250647062 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd224011af0 con 0x7fd23407cd30 2024-04-25T14:51:19.019 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.986+0000 7fd238a00700 1 -- 172.21.15.94:0/4250647062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 msgr2=0x7fd23407b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.986+0000 7fd238a00700 1 --2- 172.21.15.94:0/4250647062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23407b190 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2240098a0 tx=0x7fd224009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.986+0000 7fd238a00700 1 -- 172.21.15.94:0/4250647062 shutdown_connections 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.986+0000 7fd238a00700 1 --2- 172.21.15.94:0/4250647062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23407b190 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.986+0000 7fd238a00700 1 -- 172.21.15.94:0/4250647062 >> 172.21.15.94:0/4250647062 conn(0x7fd234103be0 msgr2=0x7fd234106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.986+0000 7fd238a00700 1 -- 172.21.15.94:0/4250647062 shutdown_connections 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.986+0000 7fd238a00700 1 -- 172.21.15.94:0/4250647062 wait complete. 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.987+0000 7fd238a00700 1 Processor -- start 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.988+0000 7fd238a00700 1 -- start start 2024-04-25T14:51:19.022 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.988+0000 7fd238a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.988+0000 7fd238a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd23410eaf0 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.988+0000 7fd231e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.988+0000 7fd231e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60782/0 (socket says 172.21.15.94:60782) 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.988+0000 7fd231e00700 1 -- 172.21.15.94:0/3190734523 learned_addr learned my addr 172.21.15.94:0/3190734523 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.989+0000 7fd231e00700 1 -- 172.21.15.94:0/3190734523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23419ec30 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.989+0000 7fd231e00700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23419e6f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd224011ac0 tx=0x7fd224011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.989+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd224003970 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.989+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd224003b10 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.989+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23419eec0 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.989+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd224024610 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.990+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2341a1bb0 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.991+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fd22401a9e0 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.991+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2340640a0 con 0x7fd23407cd30 2024-04-25T14:51:19.023 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.991+0000 7fd22aa00700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7fd218039560 0x7fd21803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:19.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.991+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd22404d8d0 con 0x7fd23407cd30 2024-04-25T14:51:19.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.991+0000 7fd231400700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7fd218039560 0x7fd21803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:19.025 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.992+0000 7fd231400700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7fd218039560 0x7fd21803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd220006fd0 tx=0x7fd220006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:17.997+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd22400b3e0 con 0x7fd23407cd30 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.282+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd2341a1d80 con 0x7fd23407cd30 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.335+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd22404d2f0 con 0x7fd23407cd30 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.338+0000 7fd22aa00700 1 -- 172.21.15.94:0/3190734523 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fd22402c350 con 0x7fd23407cd30 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.340+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7fd218039560 msgr2=0x7fd21803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7fd218039560 0x7fd21803ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd220006fd0 tx=0x7fd220006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 msgr2=0x7fd23419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23419e6f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd224011ac0 tx=0x7fd224011ba0 comp rx=0 tx=0).stop 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 shutdown_connections 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7fd218039560 0x7fd21803ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 --2- 172.21.15.94:0/3190734523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd23407cd30 0x7fd23419e6f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 >> 172.21.15.94:0/3190734523 conn(0x7fd234103be0 msgr2=0x7fd234104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.341+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 shutdown_connections 2024-04-25T14:51:19.026 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:18.342+0000 7fd238a00700 1 -- 172.21.15.94:0/3190734523 wait complete. 2024-04-25T14:51:19.311 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:18.995+0000 7f5e4f323200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:51:19.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:19 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3190734523' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T14:51:19.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:19 smithi094 ceph-mon[27862]: mgrmap e5: x(active, since 5s) 2024-04-25T14:51:19.639 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:19 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:19.407+0000 7f5e4f323200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:51:19.639 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:19 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:19.522+0000 7f5e4f323200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:51:20.606 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:20 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1622683524' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:51:20.613 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.637+0000 7f57d6a00700 1 Processor -- start 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.637+0000 7f57d6a00700 1 -- start start 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.637+0000 7f57d6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d01089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:20.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.637+0000 7f57d6a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0108fb0 con 0x7f57d01085c0 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.638+0000 7f57cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d01089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.638+0000 7f57cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d01089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60828/0 (socket says 172.21.15.94:60828) 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.638+0000 7f57cfe00700 1 -- 172.21.15.94:0/1624119666 learned_addr learned my addr 172.21.15.94:0/1624119666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.639+0000 7f57cfe00700 1 -- 172.21.15.94:0/1624119666 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d01097c0 con 0x7f57d01085c0 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.639+0000 7f57cfe00700 1 --2- 172.21.15.94:0/1624119666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d01089e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57c00098a0 tx=0x7f57c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc6ea78197ef6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.639+0000 7f57cea00700 1 -- 172.21.15.94:0/1624119666 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c0004160 con 0x7f57d01085c0 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.639+0000 7f57cea00700 1 -- 172.21.15.94:0/1624119666 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f57c0004300 con 0x7f57d01085c0 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.639+0000 7f57cea00700 1 -- 172.21.15.94:0/1624119666 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c0011af0 con 0x7f57d01085c0 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.641+0000 7f57d6a00700 1 -- 172.21.15.94:0/1624119666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 msgr2=0x7f57d01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.641+0000 7f57d6a00700 1 --2- 172.21.15.94:0/1624119666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d01089e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f57c00098a0 tx=0x7f57c0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.641+0000 7f57d6a00700 1 -- 172.21.15.94:0/1624119666 shutdown_connections 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.641+0000 7f57d6a00700 1 --2- 172.21.15.94:0/1624119666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d01089e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.641+0000 7f57d6a00700 1 -- 172.21.15.94:0/1624119666 >> 172.21.15.94:0/1624119666 conn(0x7f57d0103be0 msgr2=0x7f57d0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:20.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.641+0000 7f57d6a00700 1 -- 172.21.15.94:0/1624119666 shutdown_connections 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.641+0000 7f57d6a00700 1 -- 172.21.15.94:0/1624119666 wait complete. 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.642+0000 7f57d6a00700 1 Processor -- start 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.642+0000 7f57d6a00700 1 -- start start 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.643+0000 7f57d6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.643+0000 7f57d6a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f57d010c9f0 con 0x7f57d01085c0 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.643+0000 7f57cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.643+0000 7f57cfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60832/0 (socket says 172.21.15.94:60832) 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.643+0000 7f57cfe00700 1 -- 172.21.15.94:0/1622683524 learned_addr learned my addr 172.21.15.94:0/1622683524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.643+0000 7f57cfe00700 1 -- 172.21.15.94:0/1622683524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d019ca60 con 0x7f57d01085c0 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.644+0000 7f57cfe00700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d019c520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57c00117b0 tx=0x7f57c0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.644+0000 7f57ccc00700 1 -- 172.21.15.94:0/1622683524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c0003af0 con 0x7f57d01085c0 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.644+0000 7f57ccc00700 1 -- 172.21.15.94:0/1622683524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f57c0020070 con 0x7f57d01085c0 2024-04-25T14:51:20.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.644+0000 7f57ccc00700 1 -- 172.21.15.94:0/1622683524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57c0019630 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.644+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d019ccf0 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.644+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d019f9e0 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.645+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d010cb70 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.645+0000 7f57ccc00700 1 -- 172.21.15.94:0/1622683524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f57c001a9c0 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.646+0000 7f57ccc00700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 0x7f57b403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.646+0000 7f57ccc00700 1 -- 172.21.15.94:0/1622683524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f57c004dae0 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.646+0000 7f57cf400700 1 -- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 msgr2=0x7f57b403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/17546499 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.646+0000 7f57cf400700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 0x7f57b403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.655+0000 7f57ccc00700 1 -- 172.21.15.94:0/1622683524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57c000e260 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.847+0000 7f57cf400700 1 -- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 msgr2=0x7f57b403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/17546499 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.847+0000 7f57cf400700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 0x7f57b403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.939+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f57d019ff90 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.939+0000 7f57ccc00700 1 -- 172.21.15.94:0/1622683524 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f57c00042c0 con 0x7f57d01085c0 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 msgr2=0x7f57b403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:51:20.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 0x7f57b403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 msgr2=0x7f57d019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d019c520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f57c00117b0 tx=0x7f57c0011b70 comp rx=0 tx=0).stop 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 shutdown_connections 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f57b4039600 0x7f57b403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 --2- 172.21.15.94:0/1622683524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57d01085c0 0x7f57d019c520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.943+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 >> 172.21.15.94:0/1622683524 conn(0x7f57d0103be0 msgr2=0x7f57d0104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.944+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 shutdown_connections 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:19.944+0000 7f57d6a00700 1 -- 172.21.15.94:0/1622683524 wait complete. 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-04-25T14:51:20.624 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-04-25T14:51:21.158 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:21.091+0000 7f5e4f323200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:51:21.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:21.317+0000 7f5e4f323200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:51:21.913 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:21.534+0000 7f5e4f323200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:51:22.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:21.911+0000 7f5e4f323200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:51:22.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:22.033+0000 7f5e4f323200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:51:22.537 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:22.284+0000 7f5e4f323200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:51:22.961 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:22.536+0000 7f5e4f323200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:51:23.711 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:23.330+0000 7f5e4f323200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:51:24.170 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:23.882+0000 7f5e4f323200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:51:24.171 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:24.009+0000 7f5e4f323200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:51:24.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:24.275+0000 7f5e4f323200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:51:25.211 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:24.916+0000 7f5e4f323200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:51:25.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:25.038+0000 7f5e4f323200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:51:25.669 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:25.361+0000 7f5e4f323200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:51:25.670 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:25.540+0000 7f5e4f323200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:51:25.670 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:25.668+0000 7f5e4f323200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:51:25.962 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:25.821+0000 7f5e4f323200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:51:26.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:26.202+0000 7f5e4f323200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:51:27.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:26.955+0000 7f5e4f323200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:27.236+0000 7f5e4f323200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: Active manager daemon x restarted 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: Activating manager daemon x 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: mgrmap e6: x(active, starting, since 0.00917702s) 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:51:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:51:27.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:51:27.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: Manager daemon x is now available 2024-04-25T14:51:27.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:27 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:28.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: Found migration_current of "None". Setting to last migration. 2024-04-25T14:51:28.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:28.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:28.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:28.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:51:28.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:51:28.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:28.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:28.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:28 smithi094 ceph-mon[27862]: mgrmap e7: x(active, since 1.01654s) 2024-04-25T14:51:28.813 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:51:28.813 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T14:51:28.813 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:51:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:51:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.157+0000 7f8946000700 1 Processor -- start 2024-04-25T14:51:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.157+0000 7f8946000700 1 -- start start 2024-04-25T14:51:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.157+0000 7f8946000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894007b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.157+0000 7f8946000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f894007b780 con 0x7f894007cd50 2024-04-25T14:51:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.158+0000 7f893f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894007b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.158+0000 7f893f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894007b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60836/0 (socket says 172.21.15.94:60836) 2024-04-25T14:51:28.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.158+0000 7f893f400700 1 -- 172.21.15.94:0/172373675 learned_addr learned my addr 172.21.15.94:0/172373675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:28.821 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.158+0000 7f893f400700 1 -- 172.21.15.94:0/172373675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894007b900 con 0x7f894007cd50 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.159+0000 7f893f400700 1 --2- 172.21.15.94:0/172373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894007b1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89300098a0 tx=0x7f8930009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5890f17dc57aa9de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.159+0000 7f893e000700 1 -- 172.21.15.94:0/172373675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8930004160 con 0x7f894007cd50 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.159+0000 7f893e000700 1 -- 172.21.15.94:0/172373675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8930004300 con 0x7f894007cd50 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.159+0000 7f893e000700 1 -- 172.21.15.94:0/172373675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8930011af0 con 0x7f894007cd50 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.160+0000 7f8946000700 1 -- 172.21.15.94:0/172373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 msgr2=0x7f894007b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.160+0000 7f8946000700 1 --2- 172.21.15.94:0/172373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894007b1b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89300098a0 tx=0x7f8930009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.160+0000 7f8946000700 1 -- 172.21.15.94:0/172373675 shutdown_connections 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.160+0000 7f8946000700 1 --2- 172.21.15.94:0/172373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894007b1b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.160+0000 7f8946000700 1 -- 172.21.15.94:0/172373675 >> 172.21.15.94:0/172373675 conn(0x7f8940103bf0 msgr2=0x7f8940106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.160+0000 7f8946000700 1 -- 172.21.15.94:0/172373675 shutdown_connections 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.160+0000 7f8946000700 1 -- 172.21.15.94:0/172373675 wait complete. 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.161+0000 7f8946000700 1 Processor -- start 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.161+0000 7f8946000700 1 -- start start 2024-04-25T14:51:28.822 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.161+0000 7f8946000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.161+0000 7f8946000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f894010eb40 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.162+0000 7f893f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.162+0000 7f893f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60844/0 (socket says 172.21.15.94:60844) 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.162+0000 7f893f400700 1 -- 172.21.15.94:0/935722355 learned_addr learned my addr 172.21.15.94:0/935722355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.162+0000 7f893f400700 1 -- 172.21.15.94:0/935722355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f894019ebe0 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.162+0000 7f893f400700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894019e6a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8930011ac0 tx=0x7f8930011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.163+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8930003b20 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.163+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f893001a8c0 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.163+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89300248d0 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.163+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f894019ee70 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.163+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89401a1b60 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.164+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f8930022020 con 0x7f894007cd50 2024-04-25T14:51:28.823 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.164+0000 7f8937e00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.164+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 --> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f892403c1d0 con 0x7f8924039600 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.164+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f893004d450 con 0x7f894007cd50 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.164+0000 7f893ea00700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 msgr2=0x7f892403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/17546499 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.165+0000 7f893ea00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.366+0000 7f893ea00700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 msgr2=0x7f892403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/17546499 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.366+0000 7f893ea00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.767+0000 7f893ea00700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 msgr2=0x7f892403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/17546499 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:21.767+0000 7f893ea00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:22.568+0000 7f893ea00700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 msgr2=0x7f892403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/17546499 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:22.568+0000 7f893ea00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:24.169+0000 7f893ea00700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 msgr2=0x7f892403bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/17546499 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:24.169+0000 7f893ea00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:27.245+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f893002d3f0 con 0x7f894007cd50 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:27.245+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 msgr2=0x7f892403bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:51:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:27.245+0000 7f8937e00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.249+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f8930017950 con 0x7f894007cd50 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.249+0000 7f8937e00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f892403cef0 0x7f892403f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.249+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f89300468c0 con 0x7f892403cef0 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.252+0000 7f893ea00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f892403cef0 0x7f892403f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.255+0000 7f893ea00700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f892403cef0 0x7f892403f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f892c003a10 tx=0x7f892c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.256+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f89300468c0 con 0x7f892403cef0 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.264+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f894019f350 con 0x7f892403cef0 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8937e00700 1 -- 172.21.15.94:0/935722355 <== mgr.14120 v2:172.21.15.94:6800/1217751898 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f894019f350 con 0x7f892403cef0 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f892403cef0 msgr2=0x7f892403f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f892403cef0 0x7f892403f2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f892c003a10 tx=0x7f892c0092b0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 msgr2=0x7f894019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894019e6a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8930011ac0 tx=0x7f8930011ba0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.825 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 shutdown_connections 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f894007cd50 0x7f894019e6a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f892403cef0 0x7f892403f2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.265+0000 7f8946000700 1 --2- 172.21.15.94:0/935722355 >> [v2:172.21.15.94:6800/17546499,v1:172.21.15.94:6801/17546499] conn(0x7f8924039600 0x7f892403bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.266+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 >> 172.21.15.94:0/935722355 conn(0x7f8940103bf0 msgr2=0x7f89401046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.266+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 shutdown_connections 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:28.266+0000 7f8946000700 1 -- 172.21.15.94:0/935722355 wait complete. 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-04-25T14:51:28.826 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-04-25T14:51:29.384 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:29 smithi094 ceph-mon[27862]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T14:51:29.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:29 smithi094 ceph-mon[27862]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T14:51:29.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:29 smithi094 ceph-mon[27862]: [25/Apr/2024:14:51:28] ENGINE Bus STARTING 2024-04-25T14:51:29.385 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:29 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:30.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.384+0000 7fe1d3800700 1 Processor -- start 2024-04-25T14:51:30.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.384+0000 7fe1d3800700 1 -- start start 2024-04-25T14:51:30.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.384+0000 7fe1d3800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc108a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.384+0000 7fe1d3800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc109010 con 0x7fe1cc108620 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.384+0000 7fe1d0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc108a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.384+0000 7fe1d0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc108a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42646/0 (socket says 172.21.15.94:42646) 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.384+0000 7fe1d0e00700 1 -- 172.21.15.94:0/1064179215 learned_addr learned my addr 172.21.15.94:0/1064179215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.385+0000 7fe1d0e00700 1 -- 172.21.15.94:0/1064179215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc1097d0 con 0x7fe1cc108620 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.385+0000 7fe1d0e00700 1 --2- 172.21.15.94:0/1064179215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc108a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00098a0 tx=0x7fe1c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216ac40fcb31408e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.385+0000 7fe1cb400700 1 -- 172.21.15.94:0/1064179215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1c0004160 con 0x7fe1cc108620 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.386+0000 7fe1cb400700 1 -- 172.21.15.94:0/1064179215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe1c0004300 con 0x7fe1cc108620 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.386+0000 7fe1cb400700 1 -- 172.21.15.94:0/1064179215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1c0011af0 con 0x7fe1cc108620 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.386+0000 7fe1d3800700 1 -- 172.21.15.94:0/1064179215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 msgr2=0x7fe1cc108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.386+0000 7fe1d3800700 1 --2- 172.21.15.94:0/1064179215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc108a40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00098a0 tx=0x7fe1c0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.387+0000 7fe1d3800700 1 -- 172.21.15.94:0/1064179215 shutdown_connections 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.387+0000 7fe1d3800700 1 --2- 172.21.15.94:0/1064179215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc108a40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.387+0000 7fe1d3800700 1 -- 172.21.15.94:0/1064179215 >> 172.21.15.94:0/1064179215 conn(0x7fe1cc103c60 msgr2=0x7fe1cc106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.387+0000 7fe1d3800700 1 -- 172.21.15.94:0/1064179215 shutdown_connections 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.387+0000 7fe1d3800700 1 -- 172.21.15.94:0/1064179215 wait complete. 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.388+0000 7fe1d3800700 1 Processor -- start 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.388+0000 7fe1d3800700 1 -- start start 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.388+0000 7fe1d3800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc07db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.388+0000 7fe1d3800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc10ca00 con 0x7fe1cc108620 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.389+0000 7fe1d0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc07db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.389+0000 7fe1d0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc07db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42652/0 (socket says 172.21.15.94:42652) 2024-04-25T14:51:30.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.389+0000 7fe1d0e00700 1 -- 172.21.15.94:0/3962764302 learned_addr learned my addr 172.21.15.94:0/3962764302 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.389+0000 7fe1d0e00700 1 -- 172.21.15.94:0/3962764302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc07e090 con 0x7fe1cc108620 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.389+0000 7fe1d0e00700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc07db50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009fb0 tx=0x7fe1c0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.389+0000 7fe1c9600700 1 -- 172.21.15.94:0/3962764302 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1c00036a0 con 0x7fe1cc108620 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.390+0000 7fe1c9600700 1 -- 172.21.15.94:0/3962764302 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe1c001a430 con 0x7fe1cc108620 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.390+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc07c280 con 0x7fe1cc108620 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.390+0000 7fe1c9600700 1 -- 172.21.15.94:0/3962764302 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1c0019550 con 0x7fe1cc108620 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.390+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc07c7c0 con 0x7fe1cc108620 2024-04-25T14:51:30.306 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.391+0000 7fe1c9600700 1 -- 172.21.15.94:0/3962764302 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe1c001a8e0 con 0x7fe1cc108620 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.391+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc0640a0 con 0x7fe1cc108620 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.391+0000 7fe1c9600700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe1b40395b0 0x7fe1b403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.391+0000 7fe1c9600700 1 -- 172.21.15.94:0/3962764302 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe1c004d990 con 0x7fe1cc108620 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.392+0000 7fe1cbe00700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe1b40395b0 0x7fe1b403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.392+0000 7fe1cbe00700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe1b40395b0 0x7fe1b403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006fd0 tx=0x7fe1b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.398+0000 7fe1c9600700 1 -- 172.21.15.94:0/3962764302 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1c001a690 con 0x7fe1cc108620 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.644+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fe1cc104ef0 con 0x7fe1b40395b0 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.651+0000 7fe1c9600700 1 -- 172.21.15.94:0/3962764302 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe1cc104ef0 con 0x7fe1b40395b0 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.654+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe1b40395b0 msgr2=0x7fe1b403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.654+0000 7fe1d3800700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe1b40395b0 0x7fe1b403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8006fd0 tx=0x7fe1b8006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.654+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 msgr2=0x7fe1cc07db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.654+0000 7fe1d3800700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc07db50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009fb0 tx=0x7fe1c0011940 comp rx=0 tx=0).stop 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.655+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 shutdown_connections 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.655+0000 7fe1d3800700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe1b40395b0 0x7fe1b403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.307 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.655+0000 7fe1d3800700 1 --2- 172.21.15.94:0/3962764302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1cc108620 0x7fe1cc07db50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:30.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.655+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 >> 172.21.15.94:0/3962764302 conn(0x7fe1cc103c60 msgr2=0x7fe1cc1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:30.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.655+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 shutdown_connections 2024-04-25T14:51:30.308 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:29.655+0000 7fe1d3800700 1 -- 172.21.15.94:0/3962764302 wait complete. 2024-04-25T14:51:30.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:30 smithi094 ceph-mon[27862]: [25/Apr/2024:14:51:28] ENGINE Serving on https://172.21.15.94:7150 2024-04-25T14:51:30.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:30 smithi094 ceph-mon[27862]: [25/Apr/2024:14:51:28] ENGINE Serving on http://172.21.15.94:8765 2024-04-25T14:51:30.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:30 smithi094 ceph-mon[27862]: [25/Apr/2024:14:51:28] ENGINE Bus STARTED 2024-04-25T14:51:30.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:30 smithi094 ceph-mon[27862]: mgrmap e8: x(active, since 2s) 2024-04-25T14:51:30.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:30 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:30.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:30 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:31.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:31 smithi094 ceph-mon[27862]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:51:31.828 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.919+0000 7f56b9c00700 1 Processor -- start 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.919+0000 7f56b9c00700 1 -- start start 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.920+0000 7f56b9c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b410b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.920+0000 7f56b9c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56b4063d60 con 0x7f56b4109040 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.920+0000 7f56b2a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b410b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.920+0000 7f56b2a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b410b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42660/0 (socket says 172.21.15.94:42660) 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.920+0000 7f56b2a00700 1 -- 172.21.15.94:0/4020069061 learned_addr learned my addr 172.21.15.94:0/4020069061 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.920+0000 7f56b2a00700 1 -- 172.21.15.94:0/4020069061 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b410b970 con 0x7f56b4109040 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.921+0000 7f56b2a00700 1 --2- 172.21.15.94:0/4020069061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b410b430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009e80 tx=0x7f56a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abc114d17381664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.921+0000 7f56b2000700 1 -- 172.21.15.94:0/4020069061 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56a4017070 con 0x7f56b4109040 2024-04-25T14:51:31.829 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.921+0000 7f56b2000700 1 -- 172.21.15.94:0/4020069061 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f56a400b8c0 con 0x7f56b4109040 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.921+0000 7f56b2000700 1 -- 172.21.15.94:0/4020069061 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56a4004490 con 0x7f56b4109040 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.922+0000 7f56b9c00700 1 -- 172.21.15.94:0/4020069061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 msgr2=0x7f56b410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.922+0000 7f56b9c00700 1 --2- 172.21.15.94:0/4020069061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b410b430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009e80 tx=0x7f56a400b330 comp rx=0 tx=0).stop 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.922+0000 7f56b9c00700 1 -- 172.21.15.94:0/4020069061 shutdown_connections 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.922+0000 7f56b9c00700 1 --2- 172.21.15.94:0/4020069061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b410b430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.922+0000 7f56b9c00700 1 -- 172.21.15.94:0/4020069061 >> 172.21.15.94:0/4020069061 conn(0x7f56b4102ab0 msgr2=0x7f56b4104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.922+0000 7f56b9c00700 1 -- 172.21.15.94:0/4020069061 shutdown_connections 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.922+0000 7f56b9c00700 1 -- 172.21.15.94:0/4020069061 wait complete. 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.923+0000 7f56b9c00700 1 Processor -- start 2024-04-25T14:51:31.835 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.923+0000 7f56b9c00700 1 -- start start 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.923+0000 7f56b9c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.923+0000 7f56b9c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f56b410c8d0 con 0x7f56b4109040 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.924+0000 7f56b2a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.924+0000 7f56b2a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42664/0 (socket says 172.21.15.94:42664) 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.924+0000 7f56b2a00700 1 -- 172.21.15.94:0/2133534733 learned_addr learned my addr 172.21.15.94:0/2133534733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.924+0000 7f56b2a00700 1 -- 172.21.15.94:0/2133534733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b419c980 con 0x7f56b4109040 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.924+0000 7f56b2a00700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b419c440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b7b0 tx=0x7f56a400be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.925+0000 7f56ab400700 1 -- 172.21.15.94:0/2133534733 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56a4020020 con 0x7f56b4109040 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.925+0000 7f56ab400700 1 -- 172.21.15.94:0/2133534733 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f56a4003d10 con 0x7f56b4109040 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.925+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b419cc10 con 0x7f56b4109040 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.925+0000 7f56ab400700 1 -- 172.21.15.94:0/2133534733 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56a4017040 con 0x7f56b4109040 2024-04-25T14:51:31.836 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.925+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b419f900 con 0x7f56b4109040 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.926+0000 7f56ab400700 1 -- 172.21.15.94:0/2133534733 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f56a4023020 con 0x7f56b4109040 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.926+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b4069b80 con 0x7f56b4109040 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.926+0000 7f56ab400700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f5698039600 0x7f569803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.926+0000 7f56ab400700 1 -- 172.21.15.94:0/2133534733 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f56a400f050 con 0x7f56b4109040 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.927+0000 7f56abe00700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f5698039600 0x7f569803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.927+0000 7f56abe00700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f5698039600 0x7f569803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f569c006fd0 tx=0x7f569c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:30.932+0000 7f56ab400700 1 -- 172.21.15.94:0/2133534733 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56a402a020 con 0x7f56b4109040 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.159+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f56b4103c90 con 0x7f5698039600 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.160+0000 7f56ab400700 1 -- 172.21.15.94:0/2133534733 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f56b4103c90 con 0x7f5698039600 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f5698039600 msgr2=0x7f569803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f5698039600 0x7f569803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f569c006fd0 tx=0x7f569c006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 msgr2=0x7f56b419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:31.837 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b419c440 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56a400b7b0 tx=0x7f56a400be90 comp rx=0 tx=0).stop 2024-04-25T14:51:31.838 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 shutdown_connections 2024-04-25T14:51:31.838 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f5698039600 0x7f569803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.838 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 --2- 172.21.15.94:0/2133534733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f56b4109040 0x7f56b419c440 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:31.838 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.163+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 >> 172.21.15.94:0/2133534733 conn(0x7f56b4102ab0 msgr2=0x7f56b4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:31.838 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.164+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 shutdown_connections 2024-04-25T14:51:31.838 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:31.164+0000 7f56b9c00700 1 -- 172.21.15.94:0/2133534733 wait complete. 2024-04-25T14:51:31.838 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-04-25T14:51:32.412 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:32 smithi094 ceph-mon[27862]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: Generating public/private rsa key pair. 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: Your identification has been saved in /tmp/tmph6an209n/key. 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: Your public key has been saved in /tmp/tmph6an209n/key.pub. 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: The key fingerprint is: 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: SHA256:6xzGKz8OhbdNO11avG9T9QR0juvVN2pr7aQ/RamoD94 ceph-f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: The key's randomart image is: 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: +---[RSA 3072]----+ 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | .. .| 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | .+ | 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | ..o| 2024-04-25T14:51:33.598 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | . . o=| 2024-04-25T14:51:33.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | . S . . *+*| 2024-04-25T14:51:33.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | + = + *.o*| 2024-04-25T14:51:33.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | . *.= oo+.o| 2024-04-25T14:51:33.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | .=o+o...+=.| 2024-04-25T14:51:33.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: | +*o.E.oo++| 2024-04-25T14:51:33.599 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: +----[SHA256]-----+ 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.468+0000 7f2112400700 1 Processor -- start 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.468+0000 7f2112400700 1 -- start start 2024-04-25T14:51:33.853 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.469+0000 7f2112400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:33.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.469+0000 7f2112400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f210c108fd0 con 0x7f210c1085e0 2024-04-25T14:51:33.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.469+0000 7f210b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:33.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.469+0000 7f210b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42670/0 (socket says 172.21.15.94:42670) 2024-04-25T14:51:33.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.469+0000 7f210b400700 1 -- 172.21.15.94:0/1319046077 learned_addr learned my addr 172.21.15.94:0/1319046077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:33.858 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.469+0000 7f210b400700 1 -- 172.21.15.94:0/1319046077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c109790 con 0x7f210c1085e0 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.470+0000 7f210b400700 1 --2- 172.21.15.94:0/1319046077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c108a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098a0 tx=0x7f20fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722f1e7c9e3a42db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.470+0000 7f210aa00700 1 -- 172.21.15.94:0/1319046077 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20fc004160 con 0x7f210c1085e0 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.470+0000 7f210aa00700 1 -- 172.21.15.94:0/1319046077 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f20fc004300 con 0x7f210c1085e0 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.470+0000 7f210aa00700 1 -- 172.21.15.94:0/1319046077 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20fc011af0 con 0x7f210c1085e0 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.471+0000 7f2112400700 1 -- 172.21.15.94:0/1319046077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 msgr2=0x7f210c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.471+0000 7f2112400700 1 --2- 172.21.15.94:0/1319046077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c108a00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098a0 tx=0x7f20fc009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.471+0000 7f2112400700 1 -- 172.21.15.94:0/1319046077 shutdown_connections 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.471+0000 7f2112400700 1 --2- 172.21.15.94:0/1319046077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c108a00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.471+0000 7f2112400700 1 -- 172.21.15.94:0/1319046077 >> 172.21.15.94:0/1319046077 conn(0x7f210c103c60 msgr2=0x7f210c106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.471+0000 7f2112400700 1 -- 172.21.15.94:0/1319046077 shutdown_connections 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.471+0000 7f2112400700 1 -- 172.21.15.94:0/1319046077 wait complete. 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.472+0000 7f2112400700 1 Processor -- start 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.472+0000 7f2112400700 1 -- start start 2024-04-25T14:51:33.859 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.472+0000 7f2112400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.472+0000 7f2112400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f210c10c9c0 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.473+0000 7f210b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.473+0000 7f210b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42686/0 (socket says 172.21.15.94:42686) 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.473+0000 7f210b400700 1 -- 172.21.15.94:0/956975376 learned_addr learned my addr 172.21.15.94:0/956975376 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.473+0000 7f210b400700 1 -- 172.21.15.94:0/956975376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c19ec60 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.473+0000 7f210b400700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c19e720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0117b0 tx=0x7f20fc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.474+0000 7f2108c00700 1 -- 172.21.15.94:0/956975376 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20fc003af0 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.474+0000 7f2108c00700 1 -- 172.21.15.94:0/956975376 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f20fc020070 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.474+0000 7f2108c00700 1 -- 172.21.15.94:0/956975376 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20fc019630 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.474+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c19eef0 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.474+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c1a1be0 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.475+0000 7f2108c00700 1 -- 172.21.15.94:0/956975376 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f20fc01a9c0 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.475+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210c10cb40 con 0x7f210c1085e0 2024-04-25T14:51:33.860 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.475+0000 7f2108c00700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f20f0039600 0x7f20f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.475+0000 7f2108c00700 1 -- 172.21.15.94:0/956975376 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f20fc04dbf0 con 0x7f210c1085e0 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.476+0000 7f2102a00700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f20f0039600 0x7f20f003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.476+0000 7f2102a00700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f20f0039600 0x7f20f003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006fd0 tx=0x7f20f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.481+0000 7f2108c00700 1 -- 172.21.15.94:0/956975376 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20fc0042a0 con 0x7f210c1085e0 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:32.704+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f210c104e90 con 0x7f20f0039600 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.263+0000 7f2108c00700 1 -- 172.21.15.94:0/956975376 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f210c104e90 con 0x7f20f0039600 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.266+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f20f0039600 msgr2=0x7f20f003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.266+0000 7f2112400700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f20f0039600 0x7f20f003bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20f4006fd0 tx=0x7f20f4006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.266+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 msgr2=0x7f210c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.266+0000 7f2112400700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c19e720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0117b0 tx=0x7f20fc011b70 comp rx=0 tx=0).stop 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.267+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 shutdown_connections 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.267+0000 7f2112400700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f20f0039600 0x7f20f003bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.267+0000 7f2112400700 1 --2- 172.21.15.94:0/956975376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f210c1085e0 0x7f210c19e720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:33.861 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.267+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 >> 172.21.15.94:0/956975376 conn(0x7f210c103c60 msgr2=0x7f210c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:33.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.267+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 shutdown_connections 2024-04-25T14:51:33.862 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:33.267+0000 7f2112400700 1 -- 172.21.15.94:0/956975376 wait complete. 2024-04-25T14:51:34.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:34 smithi094 ceph-mon[27862]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:51:34.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:34 smithi094 ceph-mon[27862]: Generating ssh key... 2024-04-25T14:51:34.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:34 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:34.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:34 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:35.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7bxQUxxNc1aQJNgUe/jMN3nmsh+0+kibcaPde8lWQqxZAJZ5ggU7ef9la+LvPLBwt33rJQGQUQj7R8qkDCPSsV0QRqIBnCUJXfnFfWpbQR3dsH+fiSeQM5CO+xEe7TE/FtQOIi6DAP/8kKb+rTin9iUMiDRRPsaAwFDzdC5dA0o4ua32A4m+E2bYKJxe5uYN35VbTQN8IqwgeITIB8P7lCqR+rvmoVy1V62TC11xfqtPuPGd9TIhphnahF2PI4WTLaabHWrJonow3J/+ZJdrUKNdi3s+fEBN0zIZAE7KBCxy/OCtXxC+jKMXs6cOKWlE0nkExPxHT9yxocXO3ZKyGiw1rsGcHEl/93qlO0zu+ibiCQjybuOhloc7Gy3W3QEj6K5ig47w/xQjzRBpJi1PusKFbjff8n7Mf4UT3LAV0IO7UkYpE44b5Y1+uzpoHI6gLBwPl9IJmKAk2YokZlygT0QdmeKDe848NY2PUiM75ml/iZMqI22zoNxAqGAiUrDU= ceph-f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.515+0000 7f2bd7200700 1 Processor -- start 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.515+0000 7f2bd7200700 1 -- start start 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.516+0000 7f2bd7200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd007b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.516+0000 7f2bd7200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd007b780 con 0x7f2bd007cd50 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.516+0000 7f2bcfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd007b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.516+0000 7f2bcfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd007b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42690/0 (socket says 172.21.15.94:42690) 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.516+0000 7f2bcfe00700 1 -- 172.21.15.94:0/2391758659 learned_addr learned my addr 172.21.15.94:0/2391758659 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.517+0000 7f2bcfe00700 1 -- 172.21.15.94:0/2391758659 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd007b900 con 0x7f2bd007cd50 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.517+0000 7f2bcfe00700 1 --2- 172.21.15.94:0/2391758659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd007b1b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00098a0 tx=0x7f2bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d09420de17c0a8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.517+0000 7f2bcf400700 1 -- 172.21.15.94:0/2391758659 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bc0004160 con 0x7f2bd007cd50 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.517+0000 7f2bcf400700 1 -- 172.21.15.94:0/2391758659 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2bc0004300 con 0x7f2bd007cd50 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bcf400700 1 -- 172.21.15.94:0/2391758659 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bc0011af0 con 0x7f2bd007cd50 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bd7200700 1 -- 172.21.15.94:0/2391758659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 msgr2=0x7f2bd007b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:35.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bd7200700 1 --2- 172.21.15.94:0/2391758659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd007b1b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00098a0 tx=0x7f2bc0009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bd7200700 1 -- 172.21.15.94:0/2391758659 shutdown_connections 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bd7200700 1 --2- 172.21.15.94:0/2391758659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd007b1b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bd7200700 1 -- 172.21.15.94:0/2391758659 >> 172.21.15.94:0/2391758659 conn(0x7f2bd0103be0 msgr2=0x7f2bd0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bd7200700 1 -- 172.21.15.94:0/2391758659 shutdown_connections 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.518+0000 7f2bd7200700 1 -- 172.21.15.94:0/2391758659 wait complete. 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.519+0000 7f2bd7200700 1 Processor -- start 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.520+0000 7f2bd7200700 1 -- start start 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.520+0000 7f2bd7200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.520+0000 7f2bd7200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd010eb30 con 0x7f2bd007cd50 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.520+0000 7f2bcfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.520+0000 7f2bcfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42706/0 (socket says 172.21.15.94:42706) 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.520+0000 7f2bcfe00700 1 -- 172.21.15.94:0/1840862405 learned_addr learned my addr 172.21.15.94:0/1840862405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.520+0000 7f2bcfe00700 1 -- 172.21.15.94:0/1840862405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd019ec20 con 0x7f2bd007cd50 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.521+0000 7f2bcfe00700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd019e6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0011ac0 tx=0x7f2bc0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:35.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.521+0000 7f2bcd600700 1 -- 172.21.15.94:0/1840862405 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bc0003970 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.521+0000 7f2bcd600700 1 -- 172.21.15.94:0/1840862405 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2bc0003b10 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.521+0000 7f2bcd600700 1 -- 172.21.15.94:0/1840862405 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bc0024610 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.521+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd019eeb0 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.521+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a1ba0 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.522+0000 7f2bcd600700 1 -- 172.21.15.94:0/1840862405 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2bc001a9e0 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.522+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd00640a0 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.523+0000 7f2bcd600700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2bb4039600 0x7f2bb403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.523+0000 7f2bcd600700 1 -- 172.21.15.94:0/1840862405 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2bc004d980 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.523+0000 7f2bc7e00700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2bb4039600 0x7f2bb403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.524+0000 7f2bc7e00700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2bb4039600 0x7f2bb403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006fd0 tx=0x7f2bb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.528+0000 7f2bcd600700 1 -- 172.21.15.94:0/1840862405 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bc000b3e0 con 0x7f2bd007cd50 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.755+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2bd0104e80 con 0x7f2bb4039600 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.756+0000 7f2bcd600700 1 -- 172.21.15.94:0/1840862405 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2bd0104e80 con 0x7f2bb4039600 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2bb4039600 msgr2=0x7f2bb403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:35.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2bb4039600 0x7f2bb403bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006fd0 tx=0x7f2bb8006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 msgr2=0x7f2bd019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd019e6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0011ac0 tx=0x7f2bc0011ba0 comp rx=0 tx=0).stop 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 shutdown_connections 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2bb4039600 0x7f2bb403bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 --2- 172.21.15.94:0/1840862405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bd007cd50 0x7f2bd019e6e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 >> 172.21.15.94:0/1840862405 conn(0x7f2bd0103be0 msgr2=0x7f2bd0104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.759+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 shutdown_connections 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:34.760+0000 7f2bd7200700 1 -- 172.21.15.94:0/1840862405 wait complete. 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T14:51:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T14:51:35.446 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-04-25T14:51:35.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:35 smithi094 ceph-mon[27862]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:51:36.634 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:36 smithi094 ceph-mon[27862]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:51:38.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:38 smithi094 ceph-mon[27862]: Deploying cephadm binary to smithi094 2024-04-25T14:51:42.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.027+0000 7f41f0a00700 1 Processor -- start 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.027+0000 7f41f0a00700 1 -- start start 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.028+0000 7f41f0a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec10f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.028+0000 7f41f0a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec10fa30 con 0x7f41ec10f040 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.028+0000 7f41e9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.028+0000 7f41e9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec10f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42718/0 (socket says 172.21.15.94:42718) 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.028+0000 7f41e9e00700 1 -- 172.21.15.94:0/2406169847 learned_addr learned my addr 172.21.15.94:0/2406169847 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.028+0000 7f41e9e00700 1 -- 172.21.15.94:0/2406169847 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec10fdc0 con 0x7f41ec10f040 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.029+0000 7f41e9e00700 1 --2- 172.21.15.94:0/2406169847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec10f460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0098a0 tx=0x7f41dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c850218eeb0259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.029+0000 7f41e8a00700 1 -- 172.21.15.94:0/2406169847 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc004160 con 0x7f41ec10f040 2024-04-25T14:51:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.029+0000 7f41e8a00700 1 -- 172.21.15.94:0/2406169847 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f41dc004300 con 0x7f41ec10f040 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.029+0000 7f41e8a00700 1 -- 172.21.15.94:0/2406169847 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc011af0 con 0x7f41ec10f040 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.030+0000 7f41f0a00700 1 -- 172.21.15.94:0/2406169847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 msgr2=0x7f41ec10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.030+0000 7f41f0a00700 1 --2- 172.21.15.94:0/2406169847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec10f460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0098a0 tx=0x7f41dc009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.030+0000 7f41f0a00700 1 -- 172.21.15.94:0/2406169847 shutdown_connections 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.030+0000 7f41f0a00700 1 --2- 172.21.15.94:0/2406169847 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec10f460 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.030+0000 7f41f0a00700 1 -- 172.21.15.94:0/2406169847 >> 172.21.15.94:0/2406169847 conn(0x7f41ec07ddf0 msgr2=0x7f41ec080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.030+0000 7f41f0a00700 1 -- 172.21.15.94:0/2406169847 shutdown_connections 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.030+0000 7f41f0a00700 1 -- 172.21.15.94:0/2406169847 wait complete. 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.031+0000 7f41f0a00700 1 Processor -- start 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.031+0000 7f41f0a00700 1 -- start start 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.032+0000 7f41f0a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec1a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.032+0000 7f41f0a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec112ff0 con 0x7f41ec10f040 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.032+0000 7f41e9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec1a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.032+0000 7f41e9e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec1a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42734/0 (socket says 172.21.15.94:42734) 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.032+0000 7f41e9e00700 1 -- 172.21.15.94:0/2119991907 learned_addr learned my addr 172.21.15.94:0/2119991907 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.032+0000 7f41e9e00700 1 -- 172.21.15.94:0/2119991907 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec1a2fa0 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.033+0000 7f41e9e00700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec1a2a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41dc011ac0 tx=0x7f41dc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.033+0000 7f41e2a00700 1 -- 172.21.15.94:0/2119991907 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc003b20 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.033+0000 7f41e2a00700 1 -- 172.21.15.94:0/2119991907 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f41dc01a8c0 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.033+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec1a3230 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.033+0000 7f41e2a00700 1 -- 172.21.15.94:0/2119991907 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41dc0248d0 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.033+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec1a5f20 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.035+0000 7f41e2a00700 1 -- 172.21.15.94:0/2119991907 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f41dc022020 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.035+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec113170 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.035+0000 7f41e2a00700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f41d0039600 0x7f41d003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.035+0000 7f41e2a00700 1 -- 172.21.15.94:0/2119991907 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f41dc04d810 con 0x7f41ec10f040 2024-04-25T14:51:42.792 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.035+0000 7f41e9400700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f41d0039600 0x7f41d003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.036+0000 7f41e9400700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f41d0039600 0x7f41d003bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006fd0 tx=0x7f41d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.040+0000 7f41e2a00700 1 -- 172.21.15.94:0/2119991907 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41dc01ab20 con 0x7f41ec10f040 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:36.267+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}) -- 0x7f41ec07ffb0 con 0x7f41d0039600 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.073+0000 7f41e2a00700 1 -- 172.21.15.94:0/2119991907 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f41ec07ffb0 con 0x7f41d0039600 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.077+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f41d0039600 msgr2=0x7f41d003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.077+0000 7f41f0a00700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f41d0039600 0x7f41d003bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f41d8006fd0 tx=0x7f41d8006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.077+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 msgr2=0x7f41ec1a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.077+0000 7f41f0a00700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec1a2a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f41dc011ac0 tx=0x7f41dc011ba0 comp rx=0 tx=0).stop 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.078+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 shutdown_connections 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.078+0000 7f41f0a00700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f41d0039600 0x7f41d003bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.078+0000 7f41f0a00700 1 --2- 172.21.15.94:0/2119991907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41ec10f040 0x7f41ec1a2a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.078+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 >> 172.21.15.94:0/2119991907 conn(0x7f41ec07ddf0 msgr2=0x7f41ec07f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.078+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 shutdown_connections 2024-04-25T14:51:42.793 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:42.078+0000 7f41f0a00700 1 -- 172.21.15.94:0/2119991907 wait complete. 2024-04-25T14:51:42.794 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mon service... 2024-04-25T14:51:43.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:43 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:43.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:43 smithi094 ceph-mon[27862]: Added host smithi094 2024-04-25T14:51:43.409 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:43 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:51:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:44 smithi094 ceph-mon[27862]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:51:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:44 smithi094 ceph-mon[27862]: Saving service mon spec with placement count:5 2024-04-25T14:51:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:44 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.420+0000 7f2ef6800700 1 Processor -- start 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.420+0000 7f2ef6800700 1 -- start start 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.420+0000 7f2ef6800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.421+0000 7f2ef6800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01109a0 con 0x7f2ef010ffb0 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.421+0000 7f2ef5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.421+0000 7f2ef5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59274/0 (socket says 172.21.15.94:59274) 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.421+0000 7f2ef5400700 1 -- 172.21.15.94:0/1812417446 learned_addr learned my addr 172.21.15.94:0/1812417446 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.421+0000 7f2ef5400700 1 -- 172.21.15.94:0/1812417446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef010ecb0 con 0x7f2ef010ffb0 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.422+0000 7f2ef5400700 1 --2- 172.21.15.94:0/1812417446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01103d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40098a0 tx=0x7f2ee4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523ff97c0329eccf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:45.085 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.422+0000 7f2eefe00700 1 -- 172.21.15.94:0/1812417446 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4004160 con 0x7f2ef010ffb0 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.422+0000 7f2eefe00700 1 -- 172.21.15.94:0/1812417446 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2ee4004300 con 0x7f2ef010ffb0 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.422+0000 7f2eefe00700 1 -- 172.21.15.94:0/1812417446 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee4011af0 con 0x7f2ef010ffb0 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.423+0000 7f2ef6800700 1 -- 172.21.15.94:0/1812417446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 msgr2=0x7f2ef01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.423+0000 7f2ef6800700 1 --2- 172.21.15.94:0/1812417446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01103d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40098a0 tx=0x7f2ee4009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.423+0000 7f2ef6800700 1 -- 172.21.15.94:0/1812417446 shutdown_connections 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.423+0000 7f2ef6800700 1 --2- 172.21.15.94:0/1812417446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01103d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.423+0000 7f2ef6800700 1 -- 172.21.15.94:0/1812417446 >> 172.21.15.94:0/1812417446 conn(0x7f2ef007dbf0 msgr2=0x7f2ef0080010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.423+0000 7f2ef6800700 1 -- 172.21.15.94:0/1812417446 shutdown_connections 2024-04-25T14:51:45.086 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.423+0000 7f2ef6800700 1 -- 172.21.15.94:0/1812417446 wait complete. 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.424+0000 7f2ef6800700 1 Processor -- start 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.424+0000 7f2ef6800700 1 -- start start 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.425+0000 7f2ef6800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.425+0000 7f2ef6800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0110b20 con 0x7f2ef010ffb0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.425+0000 7f2ef5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.425+0000 7f2ef5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59286/0 (socket says 172.21.15.94:59286) 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.425+0000 7f2ef5400700 1 -- 172.21.15.94:0/4134057844 learned_addr learned my addr 172.21.15.94:0/4134057844 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.425+0000 7f2ef5400700 1 -- 172.21.15.94:0/4134057844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef01a2e20 con 0x7f2ef010ffb0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.426+0000 7f2ef5400700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01a28e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4003f40 tx=0x7f2ee4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.426+0000 7f2eee000700 1 -- 172.21.15.94:0/4134057844 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee40038b0 con 0x7f2ef010ffb0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.426+0000 7f2eee000700 1 -- 172.21.15.94:0/4134057844 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2ee4003a50 con 0x7f2ef010ffb0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.426+0000 7f2eee000700 1 -- 172.21.15.94:0/4134057844 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ee401a5c0 con 0x7f2ef010ffb0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.426+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef01a30b0 con 0x7f2ef010ffb0 2024-04-25T14:51:45.087 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.426+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01a5da0 con 0x7f2ef010ffb0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.427+0000 7f2eee000700 1 -- 172.21.15.94:0/4134057844 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2ee4024430 con 0x7f2ef010ffb0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.427+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef0069b80 con 0x7f2ef010ffb0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.428+0000 7f2eee000700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.428+0000 7f2eee000700 1 -- 172.21.15.94:0/4134057844 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2ee404db50 con 0x7f2ef010ffb0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.428+0000 7f2ef4a00700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.429+0000 7f2ef4a00700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2ed80395b0 0x7f2ed803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.433+0000 7f2eee000700 1 -- 172.21.15.94:0/4134057844 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ee402cc00 con 0x7f2ef010ffb0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.679+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2ef007fe70 con 0x7f2ed80395b0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.687+0000 7f2eee000700 1 -- 172.21.15.94:0/4134057844 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2ef007fe70 con 0x7f2ed80395b0 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2ed80395b0 msgr2=0x7f2ed803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2ed80395b0 0x7f2ed803ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2edc006fd0 tx=0x7f2edc006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 msgr2=0x7f2ef01a28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01a28e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4003f40 tx=0x7f2ee4011890 comp rx=0 tx=0).stop 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 shutdown_connections 2024-04-25T14:51:45.088 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f2ed80395b0 0x7f2ed803ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 --2- 172.21.15.94:0/4134057844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ef010ffb0 0x7f2ef01a28e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.690+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 >> 172.21.15.94:0/4134057844 conn(0x7f2ef007dbf0 msgr2=0x7f2ef007f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.691+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 shutdown_connections 2024-04-25T14:51:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:43.691+0000 7f2ef6800700 1 -- 172.21.15.94:0/4134057844 wait complete. 2024-04-25T14:51:45.089 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mgr service... 2024-04-25T14:51:46.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:46 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:46.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:46 smithi094 ceph-mon[27862]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:51:46.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:46 smithi094 ceph-mon[27862]: Saving service mgr spec with placement count:2 2024-04-25T14:51:46.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:46 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:46.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:46 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.012+0000 7f7715c00700 1 Processor -- start 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.012+0000 7f7715c00700 1 -- start start 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.013+0000 7f7715c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771007b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.013+0000 7f7715c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f771007b760 con 0x7f771007cd30 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.013+0000 7f770ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771007b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.013+0000 7f770ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771007b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59290/0 (socket says 172.21.15.94:59290) 2024-04-25T14:51:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.013+0000 7f770ea00700 1 -- 172.21.15.94:0/2664142567 learned_addr learned my addr 172.21.15.94:0/2664142567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.014+0000 7f770ea00700 1 -- 172.21.15.94:0/2664142567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771007b8e0 con 0x7f771007cd30 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.014+0000 7f770ea00700 1 --2- 172.21.15.94:0/2664142567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771007b190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7700009e80 tx=0x7f770000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffde5682bd4f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.015+0000 7f770e000700 1 -- 172.21.15.94:0/2664142567 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7700017070 con 0x7f771007cd30 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.015+0000 7f770e000700 1 -- 172.21.15.94:0/2664142567 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f770000b8c0 con 0x7f771007cd30 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.015+0000 7f770e000700 1 -- 172.21.15.94:0/2664142567 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7700004490 con 0x7f771007cd30 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.016+0000 7f7715c00700 1 -- 172.21.15.94:0/2664142567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 msgr2=0x7f771007b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.016+0000 7f7715c00700 1 --2- 172.21.15.94:0/2664142567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771007b190 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7700009e80 tx=0x7f770000b330 comp rx=0 tx=0).stop 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.016+0000 7f7715c00700 1 -- 172.21.15.94:0/2664142567 shutdown_connections 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.016+0000 7f7715c00700 1 --2- 172.21.15.94:0/2664142567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771007b190 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.016+0000 7f7715c00700 1 -- 172.21.15.94:0/2664142567 >> 172.21.15.94:0/2664142567 conn(0x7f7710103be0 msgr2=0x7f7710106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.016+0000 7f7715c00700 1 -- 172.21.15.94:0/2664142567 shutdown_connections 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.016+0000 7f7715c00700 1 -- 172.21.15.94:0/2664142567 wait complete. 2024-04-25T14:51:46.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.017+0000 7f7715c00700 1 Processor -- start 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.017+0000 7f7715c00700 1 -- start start 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.018+0000 7f7715c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.018+0000 7f7715c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f771010eba0 con 0x7f771007cd30 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.018+0000 7f770ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.018+0000 7f770ea00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59298/0 (socket says 172.21.15.94:59298) 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.018+0000 7f770ea00700 1 -- 172.21.15.94:0/2481384458 learned_addr learned my addr 172.21.15.94:0/2481384458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.018+0000 7f770ea00700 1 -- 172.21.15.94:0/2481384458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771019ebe0 con 0x7f771007cd30 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.019+0000 7f770ea00700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771019e6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f770000b7b0 tx=0x7f770000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.019+0000 7f7707400700 1 -- 172.21.15.94:0/2481384458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7700020020 con 0x7f771007cd30 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.019+0000 7f7707400700 1 -- 172.21.15.94:0/2481384458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7700003c60 con 0x7f771007cd30 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.019+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771019ee70 con 0x7f771007cd30 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.019+0000 7f7707400700 1 -- 172.21.15.94:0/2481384458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7700017040 con 0x7f771007cd30 2024-04-25T14:51:46.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.019+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101a1b60 con 0x7f771007cd30 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.020+0000 7f7707400700 1 -- 172.21.15.94:0/2481384458 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7700023020 con 0x7f771007cd30 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.020+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77100640a0 con 0x7f771007cd30 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.021+0000 7f7707400700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f76f40395b0 0x7f76f403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.021+0000 7f7707400700 1 -- 172.21.15.94:0/2481384458 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f770000f050 con 0x7f771007cd30 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.021+0000 7f7707e00700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f76f40395b0 0x7f76f403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.022+0000 7f7707e00700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f76f40395b0 0x7f76f403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f76f8006fd0 tx=0x7f76f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.031+0000 7f7707400700 1 -- 172.21.15.94:0/2481384458 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f770002a020 con 0x7f771007cd30 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.285+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7710104e40 con 0x7f76f40395b0 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.292+0000 7f7707400700 1 -- 172.21.15.94:0/2481384458 <== mgr.14120 v2:172.21.15.94:6800/1217751898 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7710104e40 con 0x7f76f40395b0 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f76f40395b0 msgr2=0x7f76f403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f76f40395b0 0x7f76f403ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f76f8006fd0 tx=0x7f76f8006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 msgr2=0x7f771019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:46.970 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771019e6a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f770000b7b0 tx=0x7f770000bd50 comp rx=0 tx=0).stop 2024-04-25T14:51:46.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 shutdown_connections 2024-04-25T14:51:46.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f76f40395b0 0x7f76f403ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:46.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 --2- 172.21.15.94:0/2481384458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f771007cd30 0x7f771019e6a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:46.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 >> 172.21.15.94:0/2481384458 conn(0x7f7710103be0 msgr2=0x7f7710104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:46.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 shutdown_connections 2024-04-25T14:51:46.971 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:46.295+0000 7f7715c00700 1 -- 172.21.15.94:0/2481384458 wait complete. 2024-04-25T14:51:48.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:48 smithi094 ceph-mon[27862]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:51:48.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:48 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/160086574' entity='client.admin' 2024-04-25T14:51:48.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.534+0000 7f60bde00700 1 Processor -- start 2024-04-25T14:51:48.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.534+0000 7f60bde00700 1 -- start start 2024-04-25T14:51:48.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.534+0000 7f60bde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b81089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:48.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.534+0000 7f60bde00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60b8108fb0 con 0x7f60b81085c0 2024-04-25T14:51:48.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.535+0000 7f60b6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b81089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:48.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.535+0000 7f60b6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b81089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46418/0 (socket says 172.21.15.94:46418) 2024-04-25T14:51:48.749 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.535+0000 7f60b6a00700 1 -- 172.21.15.94:0/2740303077 learned_addr learned my addr 172.21.15.94:0/2740303077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.535+0000 7f60b6a00700 1 -- 172.21.15.94:0/2740303077 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b81097c0 con 0x7f60b81085c0 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.536+0000 7f60b6a00700 1 --2- 172.21.15.94:0/2740303077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b81089e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60a80098a0 tx=0x7f60a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef163c3b7d4e685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.536+0000 7f60b5600700 1 -- 172.21.15.94:0/2740303077 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a8004160 con 0x7f60b81085c0 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.536+0000 7f60b5600700 1 -- 172.21.15.94:0/2740303077 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a8004300 con 0x7f60b81085c0 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.536+0000 7f60b5600700 1 -- 172.21.15.94:0/2740303077 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a8011a20 con 0x7f60b81085c0 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.537+0000 7f60bde00700 1 -- 172.21.15.94:0/2740303077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 msgr2=0x7f60b81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.537+0000 7f60bde00700 1 --2- 172.21.15.94:0/2740303077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b81089e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f60a80098a0 tx=0x7f60a8009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.537+0000 7f60bde00700 1 -- 172.21.15.94:0/2740303077 shutdown_connections 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.537+0000 7f60bde00700 1 --2- 172.21.15.94:0/2740303077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b81089e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.537+0000 7f60bde00700 1 -- 172.21.15.94:0/2740303077 >> 172.21.15.94:0/2740303077 conn(0x7f60b8103be0 msgr2=0x7f60b8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.537+0000 7f60bde00700 1 -- 172.21.15.94:0/2740303077 shutdown_connections 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.537+0000 7f60bde00700 1 -- 172.21.15.94:0/2740303077 wait complete. 2024-04-25T14:51:48.750 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.538+0000 7f60bde00700 1 Processor -- start 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.538+0000 7f60bde00700 1 -- start start 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.539+0000 7f60bde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b819c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.539+0000 7f60bde00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60b810cb70 con 0x7f60b81085c0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.539+0000 7f60b6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b819c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.539+0000 7f60b6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b819c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46420/0 (socket says 172.21.15.94:46420) 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.539+0000 7f60b6a00700 1 -- 172.21.15.94:0/160086574 learned_addr learned my addr 172.21.15.94:0/160086574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.539+0000 7f60b6a00700 1 -- 172.21.15.94:0/160086574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b819c9c0 con 0x7f60b81085c0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.540+0000 7f60b6a00700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b819c480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60a80116e0 tx=0x7f60a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.540+0000 7f60bca00700 1 -- 172.21.15.94:0/160086574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a80039e0 con 0x7f60b81085c0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.540+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b819cc50 con 0x7f60b81085c0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.540+0000 7f60bca00700 1 -- 172.21.15.94:0/160086574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a801b8f0 con 0x7f60b81085c0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.540+0000 7f60bca00700 1 -- 172.21.15.94:0/160086574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f60a8023760 con 0x7f60b81085c0 2024-04-25T14:51:48.751 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.540+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b819f940 con 0x7f60b81085c0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.541+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b810ccf0 con 0x7f60b81085c0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.542+0000 7f60bca00700 1 -- 172.21.15.94:0/160086574 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f60a80239e0 con 0x7f60b81085c0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.542+0000 7f60bca00700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f609c0395b0 0x7f609c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.542+0000 7f60bca00700 1 -- 172.21.15.94:0/160086574 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f60a804dfe0 con 0x7f60b81085c0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.543+0000 7f60b6000700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f609c0395b0 0x7f609c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.544+0000 7f60b6000700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f609c0395b0 0x7f609c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60a4006fd0 tx=0x7f60a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.548+0000 7f60bca00700 1 -- 172.21.15.94:0/160086574 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60a801f030 con 0x7f60b81085c0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.798+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f60b819fb10 con 0x7f60b81085c0 2024-04-25T14:51:48.752 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.802+0000 7f60bca00700 1 -- 172.21.15.94:0/160086574 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f60a801bd50 con 0x7f60b81085c0 2024-04-25T14:51:48.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.804+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f609c0395b0 msgr2=0x7f609c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:48.753 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.804+0000 7f60bde00700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f609c0395b0 0x7f609c03ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60a4006fd0 tx=0x7f60a4006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.804+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 msgr2=0x7f60b819c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.804+0000 7f60bde00700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b819c480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f60a80116e0 tx=0x7f60a8011aa0 comp rx=0 tx=0).stop 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.805+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 shutdown_connections 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.805+0000 7f60bde00700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f609c0395b0 0x7f609c03ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.805+0000 7f60bde00700 1 --2- 172.21.15.94:0/160086574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60b81085c0 0x7f60b819c480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.805+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 >> 172.21.15.94:0/160086574 conn(0x7f60b8103be0 msgr2=0x7f60b81046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.805+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 shutdown_connections 2024-04-25T14:51:48.754 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:47.805+0000 7f60bde00700 1 -- 172.21.15.94:0/160086574 wait complete. 2024-04-25T14:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.590+0000 7fe56cc00700 1 Processor -- start 2024-04-25T14:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.590+0000 7fe56cc00700 1 -- start start 2024-04-25T14:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.590+0000 7fe56cc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe56807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:50.571 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.590+0000 7fe56cc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe56807c8f0 con 0x7fe56807de70 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.591+0000 7fe565e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe56807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.591+0000 7fe565e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe56807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46422/0 (socket says 172.21.15.94:46422) 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.591+0000 7fe565e00700 1 -- 172.21.15.94:0/900498676 learned_addr learned my addr 172.21.15.94:0/900498676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.592+0000 7fe565e00700 1 -- 172.21.15.94:0/900498676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56807ca70 con 0x7fe56807de70 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.592+0000 7fe565e00700 1 --2- 172.21.15.94:0/900498676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe56807c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5580098a0 tx=0x7fe558009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae159c136136aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.592+0000 7fe565400700 1 -- 172.21.15.94:0/900498676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558004160 con 0x7fe56807de70 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.592+0000 7fe565400700 1 -- 172.21.15.94:0/900498676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe558004300 con 0x7fe56807de70 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.592+0000 7fe565400700 1 -- 172.21.15.94:0/900498676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558011a20 con 0x7fe56807de70 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.593+0000 7fe56cc00700 1 -- 172.21.15.94:0/900498676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 msgr2=0x7fe56807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.593+0000 7fe56cc00700 1 --2- 172.21.15.94:0/900498676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe56807c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5580098a0 tx=0x7fe558009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.593+0000 7fe56cc00700 1 -- 172.21.15.94:0/900498676 shutdown_connections 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.593+0000 7fe56cc00700 1 --2- 172.21.15.94:0/900498676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe56807c320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.593+0000 7fe56cc00700 1 -- 172.21.15.94:0/900498676 >> 172.21.15.94:0/900498676 conn(0x7fe568103c30 msgr2=0x7fe568106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.594+0000 7fe56cc00700 1 -- 172.21.15.94:0/900498676 shutdown_connections 2024-04-25T14:51:50.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.594+0000 7fe56cc00700 1 -- 172.21.15.94:0/900498676 wait complete. 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.595+0000 7fe56cc00700 1 Processor -- start 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.595+0000 7fe56cc00700 1 -- start start 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.595+0000 7fe56cc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe5681a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.595+0000 7fe56cc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe56810a780 con 0x7fe56807de70 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.596+0000 7fe565e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe5681a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.596+0000 7fe565e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe5681a29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46432/0 (socket says 172.21.15.94:46432) 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.596+0000 7fe565e00700 1 -- 172.21.15.94:0/1892150007 learned_addr learned my addr 172.21.15.94:0/1892150007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.596+0000 7fe565e00700 1 -- 172.21.15.94:0/1892150007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5681a2ee0 con 0x7fe56807de70 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.596+0000 7fe565e00700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe5681a29a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe558000c00 tx=0x7fe558011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.597+0000 7fe55f400700 1 -- 172.21.15.94:0/1892150007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558003aa0 con 0x7fe56807de70 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.597+0000 7fe55f400700 1 -- 172.21.15.94:0/1892150007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55801b8f0 con 0x7fe56807de70 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.597+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5681a3170 con 0x7fe56807de70 2024-04-25T14:51:50.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.597+0000 7fe55f400700 1 -- 172.21.15.94:0/1892150007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558023870 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.597+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5681a5e60 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.598+0000 7fe55f400700 1 -- 172.21.15.94:0/1892150007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe558023b70 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.598+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5680640a0 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.599+0000 7fe55f400700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe54c0395b0 0x7fe54c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.599+0000 7fe55f400700 1 -- 172.21.15.94:0/1892150007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe55804df30 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.599+0000 7fe55d400700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe54c0395b0 0x7fe54c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.600+0000 7fe55d400700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe54c0395b0 0x7fe54c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe550006fd0 tx=0x7fe550006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.604+0000 7fe55f400700 1 -- 172.21.15.94:0/1892150007 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe55801f030 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.834+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fe56802e1b0 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.838+0000 7fe55f400700 1 -- 172.21.15.94:0/1892150007 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fe558048020 con 0x7fe56807de70 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe54c0395b0 msgr2=0x7fe54c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:50.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe54c0395b0 0x7fe54c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe550006fd0 tx=0x7fe550006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 msgr2=0x7fe5681a29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe5681a29a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe558000c00 tx=0x7fe558011aa0 comp rx=0 tx=0).stop 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 shutdown_connections 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7fe54c0395b0 0x7fe54c03ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 --2- 172.21.15.94:0/1892150007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe56807de70 0x7fe5681a29a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 >> 172.21.15.94:0/1892150007 conn(0x7fe568103c30 msgr2=0x7fe5681998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 shutdown_connections 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:49.841+0000 7fe56cc00700 1 -- 172.21.15.94:0/1892150007 wait complete. 2024-04-25T14:51:50.577 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-04-25T14:51:50.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:50 smithi094 ceph-mon[27862]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:51:50.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:50 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:50.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:50 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1892150007' entity='client.admin' 2024-04-25T14:51:52.561 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:52 smithi094 ceph-mon[27862]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:51:52.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:52 smithi094 ceph-mon[27862]: from='mgr.14120 172.21.15.94:0/2238567949' entity='mgr.x' 2024-04-25T14:51:52.562 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:52 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2325256118' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T14:51:52.835 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:52 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: ignoring --setuser ceph since I am not root 2024-04-25T14:51:52.835 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:52 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: ignoring --setgroup ceph since I am not root 2024-04-25T14:51:52.835 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:52 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:52.755+0000 7fefcca4f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:51:53.089 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:53.006+0000 7fefcca4f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:51:53.403 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:53.140+0000 7fefcca4f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:51:53.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:53 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2325256118' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T14:51:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:53 smithi094 ceph-mon[27862]: mgrmap e9: x(active, since 25s) 2024-04-25T14:51:53.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:53.534+0000 7fefcca4f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:51:53.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:53.646+0000 7fefcca4f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:51:54.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.221+0000 7ff2a2400700 1 Processor -- start 2024-04-25T14:51:54.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.221+0000 7ff2a2400700 1 -- start start 2024-04-25T14:51:54.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.222+0000 7ff2a2400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c10b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:54.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.222+0000 7ff2a2400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c07be30 con 0x7ff29c1090a0 2024-04-25T14:51:54.006 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.222+0000 7ff29b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.222+0000 7ff29b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46444/0 (socket says 172.21.15.94:46444) 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.222+0000 7ff29b400700 1 -- 172.21.15.94:0/3715791160 learned_addr learned my addr 172.21.15.94:0/3715791160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.223+0000 7ff29b400700 1 -- 172.21.15.94:0/3715791160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c10b9d0 con 0x7ff29c1090a0 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.223+0000 7ff29b400700 1 --2- 172.21.15.94:0/3715791160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c10b490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0098a0 tx=0x7ff28c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa25da8badceff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.223+0000 7ff29a000700 1 -- 172.21.15.94:0/3715791160 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff28c004160 con 0x7ff29c1090a0 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.223+0000 7ff29a000700 1 -- 172.21.15.94:0/3715791160 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28c004300 con 0x7ff29c1090a0 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.223+0000 7ff29a000700 1 -- 172.21.15.94:0/3715791160 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff28c011a20 con 0x7ff29c1090a0 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.224+0000 7ff2a2400700 1 -- 172.21.15.94:0/3715791160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 msgr2=0x7ff29c10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.224+0000 7ff2a2400700 1 --2- 172.21.15.94:0/3715791160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c10b490 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0098a0 tx=0x7ff28c009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.224+0000 7ff2a2400700 1 -- 172.21.15.94:0/3715791160 shutdown_connections 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.224+0000 7ff2a2400700 1 --2- 172.21.15.94:0/3715791160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c10b490 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.224+0000 7ff2a2400700 1 -- 172.21.15.94:0/3715791160 >> 172.21.15.94:0/3715791160 conn(0x7ff29c102a70 msgr2=0x7ff29c104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.224+0000 7ff2a2400700 1 -- 172.21.15.94:0/3715791160 shutdown_connections 2024-04-25T14:51:54.007 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.224+0000 7ff2a2400700 1 -- 172.21.15.94:0/3715791160 wait complete. 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.226+0000 7ff2a2400700 1 Processor -- start 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.226+0000 7ff2a2400700 1 -- start start 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.227+0000 7ff2a2400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.227+0000 7ff2a2400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c10c980 con 0x7ff29c1090a0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.227+0000 7ff29b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.227+0000 7ff29b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46454/0 (socket says 172.21.15.94:46454) 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.227+0000 7ff29b400700 1 -- 172.21.15.94:0/2325256118 learned_addr learned my addr 172.21.15.94:0/2325256118 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.227+0000 7ff29b400700 1 -- 172.21.15.94:0/2325256118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c19ec00 con 0x7ff29c1090a0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.227+0000 7ff29b400700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c19e6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0116e0 tx=0x7ff28c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.228+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff28c003b20 con 0x7ff29c1090a0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.228+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff28c01b8f0 con 0x7ff29c1090a0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.228+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff28c0238c0 con 0x7ff29c1090a0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.228+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c19ee90 con 0x7ff29c1090a0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.228+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c1a1b80 con 0x7ff29c1090a0 2024-04-25T14:51:54.008 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.229+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff29c0640a0 con 0x7ff29c1090a0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.229+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff28c01baa0 con 0x7ff29c1090a0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.230+0000 7ff293e00700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7ff2800395b0 0x7ff28003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.230+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff28c04dfe0 con 0x7ff29c1090a0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.230+0000 7ff29aa00700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7ff2800395b0 0x7ff28003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.231+0000 7ff29aa00700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7ff2800395b0 0x7ff28003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff288006fd0 tx=0x7ff288006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.235+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff28c01f030 con 0x7ff29c1090a0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:51.529+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff29c1a2080 con 0x7ff29c1090a0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.438+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff28c01a3d0 con 0x7ff29c1090a0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff293e00700 1 -- 172.21.15.94:0/2325256118 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7ff28c023a60 con 0x7ff29c1090a0 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7ff2800395b0 msgr2=0x7ff28003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7ff2800395b0 0x7ff28003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff288006fd0 tx=0x7ff288006e40 comp rx=0 tx=0).stop 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 msgr2=0x7ff29c19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:54.009 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c19e6c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0116e0 tx=0x7ff28c011aa0 comp rx=0 tx=0).stop 2024-04-25T14:51:54.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 shutdown_connections 2024-04-25T14:51:54.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7ff2800395b0 0x7ff28003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:54.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 --2- 172.21.15.94:0/2325256118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff29c1090a0 0x7ff29c19e6c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:54.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 >> 172.21.15.94:0/2325256118 conn(0x7ff29c102a70 msgr2=0x7ff29c1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:54.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.444+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 shutdown_connections 2024-04-25T14:51:54.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:52.445+0000 7ff2a2400700 1 -- 172.21.15.94:0/2325256118 wait complete. 2024-04-25T14:51:55.185 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:51:54 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/388736900' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T14:51:55.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:55.202+0000 7fefcca4f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:51:55.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:55.430+0000 7fefcca4f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:51:55.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:51:55.572 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.558+0000 7f47a6a00700 1 Processor -- start 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.558+0000 7f47a6a00700 1 -- start start 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.559+0000 7f47a6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a007b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.559+0000 7f47a6a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47a007b780 con 0x7f47a007cd50 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.559+0000 7f479fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a007b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.559+0000 7f479fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a007b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46482/0 (socket says 172.21.15.94:46482) 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.559+0000 7f479fe00700 1 -- 172.21.15.94:0/3396047835 learned_addr learned my addr 172.21.15.94:0/3396047835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.560+0000 7f479fe00700 1 -- 172.21.15.94:0/3396047835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a007b900 con 0x7f47a007cd50 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.560+0000 7f479fe00700 1 --2- 172.21.15.94:0/3396047835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a007b1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47900098a0 tx=0x7f4790009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71291bc3bf190efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:55.573 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.560+0000 7f479ea00700 1 -- 172.21.15.94:0/3396047835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4790004160 con 0x7f47a007cd50 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.560+0000 7f479ea00700 1 -- 172.21.15.94:0/3396047835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4790004300 con 0x7f47a007cd50 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.560+0000 7f479ea00700 1 -- 172.21.15.94:0/3396047835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4790011a20 con 0x7f47a007cd50 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.561+0000 7f47a6a00700 1 -- 172.21.15.94:0/3396047835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 msgr2=0x7f47a007b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.561+0000 7f47a6a00700 1 --2- 172.21.15.94:0/3396047835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a007b1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47900098a0 tx=0x7f4790009bb0 comp rx=0 tx=0).stop 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.561+0000 7f47a6a00700 1 -- 172.21.15.94:0/3396047835 shutdown_connections 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.561+0000 7f47a6a00700 1 --2- 172.21.15.94:0/3396047835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a007b1b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.561+0000 7f47a6a00700 1 -- 172.21.15.94:0/3396047835 >> 172.21.15.94:0/3396047835 conn(0x7f47a0103be0 msgr2=0x7f47a0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.561+0000 7f47a6a00700 1 -- 172.21.15.94:0/3396047835 shutdown_connections 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.561+0000 7f47a6a00700 1 -- 172.21.15.94:0/3396047835 wait complete. 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.562+0000 7f47a6a00700 1 Processor -- start 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.562+0000 7f47a6a00700 1 -- start start 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.563+0000 7f47a6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.563+0000 7f47a6a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f47a010ebe0 con 0x7f47a007cd50 2024-04-25T14:51:55.574 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.563+0000 7f479fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.563+0000 7f479fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46490/0 (socket says 172.21.15.94:46490) 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.563+0000 7f479fe00700 1 -- 172.21.15.94:0/388736900 learned_addr learned my addr 172.21.15.94:0/388736900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.563+0000 7f479fe00700 1 -- 172.21.15.94:0/388736900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a019ebf0 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.564+0000 7f479fe00700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a019e6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47900116e0 tx=0x7f4790011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.564+0000 7f479cc00700 1 -- 172.21.15.94:0/388736900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4790003b20 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.564+0000 7f479cc00700 1 -- 172.21.15.94:0/388736900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479001b8f0 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.564+0000 7f479cc00700 1 -- 172.21.15.94:0/388736900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47900238c0 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.564+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a019ee80 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.564+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01a1b20 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.565+0000 7f479cc00700 1 -- 172.21.15.94:0/388736900 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f479001baa0 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.565+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a00640a0 con 0x7f47a007cd50 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.566+0000 7f479cc00700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 0x7f478403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:51:55.575 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.566+0000 7f479cc00700 1 -- 172.21.15.94:0/388736900 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f479004e110 con 0x7f47a007cd50 2024-04-25T14:51:55.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.566+0000 7f479f400700 1 -- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 msgr2=0x7f478403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1217751898 2024-04-25T14:51:55.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.566+0000 7f479f400700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 0x7f478403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:51:55.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.571+0000 7f479cc00700 1 -- 172.21.15.94:0/388736900 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f479001f030 con 0x7f47a007cd50 2024-04-25T14:51:55.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.767+0000 7f479f400700 1 -- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 msgr2=0x7f478403bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1217751898 2024-04-25T14:51:55.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.767+0000 7f479f400700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 0x7f478403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:51:55.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.853+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f47a01a1cf0 con 0x7f47a007cd50 2024-04-25T14:51:55.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.854+0000 7f479cc00700 1 -- 172.21.15.94:0/388736900 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f479000e260 con 0x7f47a007cd50 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 msgr2=0x7f478403bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 0x7f478403bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 msgr2=0x7f47a019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a019e6b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47900116e0 tx=0x7f4790011aa0 comp rx=0 tx=0).stop 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 shutdown_connections 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7f4784039610 0x7f478403bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 --2- 172.21.15.94:0/388736900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f47a007cd50 0x7f47a019e6b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 >> 172.21.15.94:0/388736900 conn(0x7f47a0103be0 msgr2=0x7f47a0104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 shutdown_connections 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:54.857+0000 7f47a6a00700 1 -- 172.21.15.94:0/388736900 wait complete. 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-04-25T14:51:55.577 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-04-25T14:51:55.838 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:55.653+0000 7fefcca4f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:51:56.113 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:56.038+0000 7fefcca4f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:51:56.411 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:56.162+0000 7fefcca4f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:51:56.662 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:56.410+0000 7fefcca4f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:51:56.662 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:56.660+0000 7fefcca4f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:51:57.711 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:57.426+0000 7fefcca4f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:51:58.354 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:57.967+0000 7fefcca4f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:51:58.354 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:58.091+0000 7fefcca4f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:51:58.355 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:58.352+0000 7fefcca4f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:51:59.445 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:58.992+0000 7fefcca4f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:51:59.445 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:59.115+0000 7fefcca4f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:51:59.445 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:59.443+0000 7fefcca4f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:51:59.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:59.619+0000 7fefcca4f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:52:00.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:59.748+0000 7fefcca4f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:52:00.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:51:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:51:59.901+0000 7fefcca4f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:52:00.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:52:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:52:00.293+0000 7fefcca4f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:52:01.340 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:52:01.048+0000 7fefcca4f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:52:01.711 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:52:01.339+0000 7fefcca4f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: Active manager daemon x restarted 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: Activating manager daemon x 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: mgrmap e10: x(active, starting, since 0.00639418s) 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T14:52:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:01 smithi094 ceph-mon[27862]: Manager daemon x is now available 2024-04-25T14:52:02.677 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:02.677 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T14:52:02.677 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T14:52:02.677 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:02 smithi094 ceph-mon[27862]: mgrmap e11: x(active, since 1.0095s) 2024-04-25T14:52:03.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-04-25T14:52:03.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T14:52:03.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.112+0000 7feb26600700 1 Processor -- start 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.112+0000 7feb26600700 1 -- start start 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.113+0000 7feb26600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb201089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.113+0000 7feb26600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb20108fb0 con 0x7feb201085c0 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.113+0000 7feb1f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb201089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.113+0000 7feb1f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb201089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46500/0 (socket says 172.21.15.94:46500) 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.113+0000 7feb1f400700 1 -- 172.21.15.94:0/192048153 learned_addr learned my addr 172.21.15.94:0/192048153 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.113+0000 7feb1f400700 1 -- 172.21.15.94:0/192048153 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb201097c0 con 0x7feb201085c0 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.114+0000 7feb1f400700 1 --2- 172.21.15.94:0/192048153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb201089e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb100098a0 tx=0x7feb10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daee75694f1cad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.114+0000 7feb1ea00700 1 -- 172.21.15.94:0/192048153 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10004160 con 0x7feb201085c0 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.114+0000 7feb1ea00700 1 -- 172.21.15.94:0/192048153 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb10004300 con 0x7feb201085c0 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.114+0000 7feb1ea00700 1 -- 172.21.15.94:0/192048153 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10011a20 con 0x7feb201085c0 2024-04-25T14:52:03.182 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.115+0000 7feb26600700 1 -- 172.21.15.94:0/192048153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 msgr2=0x7feb201089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.115+0000 7feb26600700 1 --2- 172.21.15.94:0/192048153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb201089e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb100098a0 tx=0x7feb10009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.115+0000 7feb26600700 1 -- 172.21.15.94:0/192048153 shutdown_connections 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.115+0000 7feb26600700 1 --2- 172.21.15.94:0/192048153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb201089e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.115+0000 7feb26600700 1 -- 172.21.15.94:0/192048153 >> 172.21.15.94:0/192048153 conn(0x7feb20103be0 msgr2=0x7feb20106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.115+0000 7feb26600700 1 -- 172.21.15.94:0/192048153 shutdown_connections 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.115+0000 7feb26600700 1 -- 172.21.15.94:0/192048153 wait complete. 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.116+0000 7feb26600700 1 Processor -- start 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.116+0000 7feb26600700 1 -- start start 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.117+0000 7feb26600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb2007ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.117+0000 7feb26600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb2010cb70 con 0x7feb201085c0 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.117+0000 7feb1f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb2007ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.117+0000 7feb1f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb2007ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46516/0 (socket says 172.21.15.94:46516) 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.117+0000 7feb1f400700 1 -- 172.21.15.94:0/2127928891 learned_addr learned my addr 172.21.15.94:0/2127928891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.117+0000 7feb1f400700 1 -- 172.21.15.94:0/2127928891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2007cf80 con 0x7feb201085c0 2024-04-25T14:52:03.183 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.118+0000 7feb1f400700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb2007ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb10000c00 tx=0x7feb10011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.118+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10003aa0 con 0x7feb201085c0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.118+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb1001b8f0 con 0x7feb201085c0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.118+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2007b170 con 0x7feb201085c0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.118+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10023890 con 0x7feb201085c0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.118+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2007b6b0 con 0x7feb201085c0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.119+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7feb1002c3f0 con 0x7feb201085c0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.120+0000 7feb1cc00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.120+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 --> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb0403c230 con 0x7feb04039660 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.120+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7feb1004de70 con 0x7feb201085c0 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.120+0000 7feb16a00700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 msgr2=0x7feb0403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1217751898 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.120+0000 7feb16a00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.321+0000 7feb16a00700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 msgr2=0x7feb0403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1217751898 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.321+0000 7feb16a00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.722+0000 7feb16a00700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 msgr2=0x7feb0403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1217751898 2024-04-25T14:52:03.184 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:56.722+0000 7feb16a00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:57.523+0000 7feb16a00700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 msgr2=0x7feb0403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1217751898 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:57.523+0000 7feb16a00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:59.125+0000 7feb16a00700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 msgr2=0x7feb0403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1217751898 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:51:59.125+0000 7feb16a00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:01.347+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7feb100193d0 con 0x7feb201085c0 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:01.347+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 msgr2=0x7feb0403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:01.347+0000 7feb1cc00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.348+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7feb1004f7a0 con 0x7feb201085c0 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.348+0000 7feb1cc00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb0403cff0 0x7feb0403f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.348+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feb1000be90 con 0x7feb0403cff0 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.350+0000 7feb16a00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb0403cff0 0x7feb0403f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.351+0000 7feb16a00700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb0403cff0 0x7feb0403f3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb08003a10 tx=0x7feb08009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.352+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7feb1000be90 con 0x7feb0403cff0 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.359+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7feb2010c9c0 con 0x7feb0403cff0 2024-04-25T14:52:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.359+0000 7feb1cc00700 1 -- 172.21.15.94:0/2127928891 <== mgr.14152 v2:172.21.15.94:6800/2485357518 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7feb2010c9c0 con 0x7feb0403cff0 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb0403cff0 msgr2=0x7feb0403f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb0403cff0 0x7feb0403f3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feb08003a10 tx=0x7feb08009270 comp rx=0 tx=0).stop 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 msgr2=0x7feb2007ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb2007ca40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feb10000c00 tx=0x7feb10011aa0 comp rx=0 tx=0).stop 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 shutdown_connections 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb0403cff0 0x7feb0403f3e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb201085c0 0x7feb2007ca40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 --2- 172.21.15.94:0/2127928891 >> [v2:172.21.15.94:6800/1217751898,v1:172.21.15.94:6801/1217751898] conn(0x7feb04039660 0x7feb0403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 >> 172.21.15.94:0/2127928891 conn(0x7feb20103be0 msgr2=0x7feb20104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 shutdown_connections 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:02.360+0000 7feb26600700 1 -- 172.21.15.94:0/2127928891 wait complete. 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-04-25T14:52:03.186 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-04-25T14:52:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:03 smithi094 ceph-mon[27862]: [25/Apr/2024:14:52:01] ENGINE Bus STARTING 2024-04-25T14:52:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:03 smithi094 ceph-mon[27862]: [25/Apr/2024:14:52:01] ENGINE Serving on http://172.21.15.94:8765 2024-04-25T14:52:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:03 smithi094 ceph-mon[27862]: [25/Apr/2024:14:52:02] ENGINE Serving on https://172.21.15.94:7150 2024-04-25T14:52:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:03 smithi094 ceph-mon[27862]: [25/Apr/2024:14:52:02] ENGINE Bus STARTED 2024-04-25T14:52:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:03 smithi094 ceph-mon[27862]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T14:52:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:03 smithi094 ceph-mon[27862]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T14:52:04.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:04 smithi094 ceph-mon[27862]: mgrmap e12: x(active, since 2s) 2024-04-25T14:52:04.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:05.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T14:52:05.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.163+0000 7f8139000700 1 Processor -- start 2024-04-25T14:52:05.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.164+0000 7f8139000700 1 -- start start 2024-04-25T14:52:05.410 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.164+0000 7f8139000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813410b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.164+0000 7f8139000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8134063d60 con 0x7f8134109040 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.165+0000 7f8131e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813410b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.165+0000 7f8131e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813410b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60630/0 (socket says 172.21.15.94:60630) 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.165+0000 7f8131e00700 1 -- 172.21.15.94:0/3779500079 learned_addr learned my addr 172.21.15.94:0/3779500079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.165+0000 7f8131e00700 1 -- 172.21.15.94:0/3779500079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813410b970 con 0x7f8134109040 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.165+0000 7f8131e00700 1 --2- 172.21.15.94:0/3779500079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813410b430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81240098a0 tx=0x7f8124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bef892cc6712352 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.166+0000 7f8131400700 1 -- 172.21.15.94:0/3779500079 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8124004160 con 0x7f8134109040 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.166+0000 7f8131400700 1 -- 172.21.15.94:0/3779500079 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8124004300 con 0x7f8134109040 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.166+0000 7f8131400700 1 -- 172.21.15.94:0/3779500079 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8124011a20 con 0x7f8134109040 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.166+0000 7f8139000700 1 -- 172.21.15.94:0/3779500079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 msgr2=0x7f813410b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:05.411 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.166+0000 7f8139000700 1 --2- 172.21.15.94:0/3779500079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813410b430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f81240098a0 tx=0x7f8124009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.167+0000 7f8139000700 1 -- 172.21.15.94:0/3779500079 shutdown_connections 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.167+0000 7f8139000700 1 --2- 172.21.15.94:0/3779500079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813410b430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.167+0000 7f8139000700 1 -- 172.21.15.94:0/3779500079 >> 172.21.15.94:0/3779500079 conn(0x7f8134102ab0 msgr2=0x7f8134104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.167+0000 7f8139000700 1 -- 172.21.15.94:0/3779500079 shutdown_connections 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.167+0000 7f8139000700 1 -- 172.21.15.94:0/3779500079 wait complete. 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.168+0000 7f8139000700 1 Processor -- start 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.168+0000 7f8139000700 1 -- start start 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.169+0000 7f8139000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.169+0000 7f8139000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f813410c980 con 0x7f8134109040 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.169+0000 7f8131e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.169+0000 7f8131e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60632/0 (socket says 172.21.15.94:60632) 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.169+0000 7f8131e00700 1 -- 172.21.15.94:0/1924997401 learned_addr learned my addr 172.21.15.94:0/1924997401 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.170+0000 7f8131e00700 1 -- 172.21.15.94:0/1924997401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813419c950 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.170+0000 7f8131e00700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813419c410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f8124011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.170+0000 7f812aa00700 1 -- 172.21.15.94:0/1924997401 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8124003aa0 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.170+0000 7f812aa00700 1 -- 172.21.15.94:0/1924997401 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f812401b8f0 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.170+0000 7f812aa00700 1 -- 172.21.15.94:0/1924997401 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8124023870 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.170+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813419cbe0 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.170+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813419f8d0 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.171+0000 7f812aa00700 1 -- 172.21.15.94:0/1924997401 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f8124023b70 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.172+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81340640a0 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.172+0000 7f812aa00700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8118039600 0x7f811803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.172+0000 7f812aa00700 1 -- 172.21.15.94:0/1924997401 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f812404e210 con 0x7f8134109040 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.172+0000 7f812be00700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8118039600 0x7f811803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:05.413 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.173+0000 7f812be00700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8118039600 0x7f811803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f811c006fd0 tx=0x7f811c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.179+0000 7f812aa00700 1 -- 172.21.15.94:0/1924997401 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f812401f030 con 0x7f8134109040 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.410+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f8134103c60 con 0x7f8118039600 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.663+0000 7f812aa00700 1 -- 172.21.15.94:0/1924997401 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f8134103c60 con 0x7f8118039600 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.665+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8118039600 msgr2=0x7f811803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.665+0000 7f8139000700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8118039600 0x7f811803bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f811c006fd0 tx=0x7f811c006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 msgr2=0x7f813419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813419c410 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f8124011aa0 comp rx=0 tx=0).stop 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 shutdown_connections 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8118039600 0x7f811803bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:05.414 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 --2- 172.21.15.94:0/1924997401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8134109040 0x7f813419c410 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:05.415 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 >> 172.21.15.94:0/1924997401 conn(0x7f8134102ab0 msgr2=0x7f8134103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:05.415 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 shutdown_connections 2024-04-25T14:52:05.415 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:04.666+0000 7f8139000700 1 -- 172.21.15.94:0/1924997401 wait complete. 2024-04-25T14:52:05.415 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-04-25T14:52:05.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:05 smithi094 ceph-mon[27862]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:05.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:05.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:07.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:07 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:07.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:07 smithi094 ceph-mon[27862]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:07.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:07 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:07.226 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$kEuHH8yep2OB9NAevIuxOeCM2E0rQgn6kijDLNkJuGKNLQ1KdQoHS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714056726, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.986+0000 7ff04a800700 1 Processor -- start 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.986+0000 7ff04a800700 1 -- start start 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.986+0000 7ff04a800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04410b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.987+0000 7ff04a800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff04407bdf0 con 0x7ff0441090b0 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.987+0000 7ff043400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04410b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.987+0000 7ff043400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04410b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60636/0 (socket says 172.21.15.94:60636) 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.987+0000 7ff043400700 1 -- 172.21.15.94:0/3747893554 learned_addr learned my addr 172.21.15.94:0/3747893554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.987+0000 7ff043400700 1 -- 172.21.15.94:0/3747893554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04410b9e0 con 0x7ff0441090b0 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.988+0000 7ff043400700 1 --2- 172.21.15.94:0/3747893554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04410b4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0340098a0 tx=0x7ff034009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b403a68be8a43d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.988+0000 7ff042000700 1 -- 172.21.15.94:0/3747893554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034004160 con 0x7ff0441090b0 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.988+0000 7ff042000700 1 -- 172.21.15.94:0/3747893554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff034004300 con 0x7ff0441090b0 2024-04-25T14:52:07.227 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.988+0000 7ff042000700 1 -- 172.21.15.94:0/3747893554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034011a20 con 0x7ff0441090b0 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.989+0000 7ff04a800700 1 -- 172.21.15.94:0/3747893554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 msgr2=0x7ff04410b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.989+0000 7ff04a800700 1 --2- 172.21.15.94:0/3747893554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04410b4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0340098a0 tx=0x7ff034009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.989+0000 7ff04a800700 1 -- 172.21.15.94:0/3747893554 shutdown_connections 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.989+0000 7ff04a800700 1 --2- 172.21.15.94:0/3747893554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04410b4a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.989+0000 7ff04a800700 1 -- 172.21.15.94:0/3747893554 >> 172.21.15.94:0/3747893554 conn(0x7ff044102a60 msgr2=0x7ff044104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.989+0000 7ff04a800700 1 -- 172.21.15.94:0/3747893554 shutdown_connections 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.989+0000 7ff04a800700 1 -- 172.21.15.94:0/3747893554 wait complete. 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.990+0000 7ff04a800700 1 Processor -- start 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.990+0000 7ff04a800700 1 -- start start 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.991+0000 7ff04a800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.991+0000 7ff04a800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff04410c980 con 0x7ff0441090b0 2024-04-25T14:52:07.228 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.991+0000 7ff043400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.991+0000 7ff043400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04419c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60640/0 (socket says 172.21.15.94:60640) 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.991+0000 7ff043400700 1 -- 172.21.15.94:0/2751259312 learned_addr learned my addr 172.21.15.94:0/2751259312 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.992+0000 7ff043400700 1 -- 172.21.15.94:0/2751259312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04419c910 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.992+0000 7ff043400700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04419c3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff034011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.992+0000 7ff03be00700 1 -- 172.21.15.94:0/2751259312 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034003aa0 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.992+0000 7ff03be00700 1 -- 172.21.15.94:0/2751259312 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03401b8f0 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.992+0000 7ff03be00700 1 -- 172.21.15.94:0/2751259312 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff034023870 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.992+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04419cba0 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.992+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04419f890 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.993+0000 7ff03be00700 1 -- 172.21.15.94:0/2751259312 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff034023b70 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.994+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0440640a0 con 0x7ff0441090b0 2024-04-25T14:52:07.229 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.994+0000 7ff03be00700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff028039600 0x7ff02803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.994+0000 7ff03be00700 1 -- 172.21.15.94:0/2751259312 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff03404dfc0 con 0x7ff0441090b0 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.994+0000 7ff042a00700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff028039600 0x7ff02803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:05.995+0000 7ff042a00700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff028039600 0x7ff02803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff030006fd0 tx=0x7ff030006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.000+0000 7ff03be00700 1 -- 172.21.15.94:0/2751259312 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff03401f030 con 0x7ff0441090b0 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.244+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff04419fb20 con 0x7ff028039600 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.484+0000 7ff03be00700 1 -- 172.21.15.94:0/2751259312 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff04419fb20 con 0x7ff028039600 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.486+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff028039600 msgr2=0x7ff02803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.486+0000 7ff04a800700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff028039600 0x7ff02803bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff030006fd0 tx=0x7ff030006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:07.230 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.486+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 msgr2=0x7ff04419c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.486+0000 7ff04a800700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04419c3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff034000c00 tx=0x7ff034011aa0 comp rx=0 tx=0).stop 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.486+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 shutdown_connections 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.487+0000 7ff04a800700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff028039600 0x7ff02803bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.487+0000 7ff04a800700 1 --2- 172.21.15.94:0/2751259312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0441090b0 0x7ff04419c3d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.487+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 >> 172.21.15.94:0/2751259312 conn(0x7ff044102a60 msgr2=0x7ff0441034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.487+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 shutdown_connections 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:06.487+0000 7ff04a800700 1 -- 172.21.15.94:0/2751259312 wait complete. 2024-04-25T14:52:07.231 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-04-25T14:52:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T14:52:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.074+0000 7f0563c00700 1 Processor -- start 2024-04-25T14:52:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.074+0000 7f0563c00700 1 -- start start 2024-04-25T14:52:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.075+0000 7f0563c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:09.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.075+0000 7f0563c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f055c07b7c0 con 0x7f055c07cd90 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.075+0000 7f0561200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.075+0000 7f0561200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c07b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38718/0 (socket says 172.21.15.94:38718) 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.075+0000 7f0561200700 1 -- 172.21.15.94:0/1297256924 learned_addr learned my addr 172.21.15.94:0/1297256924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.076+0000 7f0561200700 1 -- 172.21.15.94:0/1297256924 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c07b940 con 0x7f055c07cd90 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.076+0000 7f0561200700 1 --2- 172.21.15.94:0/1297256924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c07b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05500098a0 tx=0x7f0550009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce40b228b9f455a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.076+0000 7f055b400700 1 -- 172.21.15.94:0/1297256924 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0550004160 con 0x7f055c07cd90 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.076+0000 7f055b400700 1 -- 172.21.15.94:0/1297256924 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0550004300 con 0x7f055c07cd90 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.076+0000 7f055b400700 1 -- 172.21.15.94:0/1297256924 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0550011a20 con 0x7f055c07cd90 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.077+0000 7f0563c00700 1 -- 172.21.15.94:0/1297256924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 msgr2=0x7f055c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.077+0000 7f0563c00700 1 --2- 172.21.15.94:0/1297256924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c07b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f05500098a0 tx=0x7f0550009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.077+0000 7f0563c00700 1 -- 172.21.15.94:0/1297256924 shutdown_connections 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.077+0000 7f0563c00700 1 --2- 172.21.15.94:0/1297256924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c07b1f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:09.076 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.077+0000 7f0563c00700 1 -- 172.21.15.94:0/1297256924 >> 172.21.15.94:0/1297256924 conn(0x7f055c103be0 msgr2=0x7f055c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.077+0000 7f0563c00700 1 -- 172.21.15.94:0/1297256924 shutdown_connections 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.077+0000 7f0563c00700 1 -- 172.21.15.94:0/1297256924 wait complete. 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.078+0000 7f0563c00700 1 Processor -- start 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.079+0000 7f0563c00700 1 -- start start 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.079+0000 7f0563c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.079+0000 7f0563c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f055c10eba0 con 0x7f055c07cd90 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.079+0000 7f0561200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.079+0000 7f0561200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38720/0 (socket says 172.21.15.94:38720) 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.079+0000 7f0561200700 1 -- 172.21.15.94:0/2734073984 learned_addr learned my addr 172.21.15.94:0/2734073984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.079+0000 7f0561200700 1 -- 172.21.15.94:0/2734073984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c19ebf0 con 0x7f055c07cd90 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.080+0000 7f0561200700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c19e6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0550000c00 tx=0x7f0550011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.080+0000 7f0559600700 1 -- 172.21.15.94:0/2734073984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0550003aa0 con 0x7f055c07cd90 2024-04-25T14:52:09.077 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.080+0000 7f0559600700 1 -- 172.21.15.94:0/2734073984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055001b8f0 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.080+0000 7f0559600700 1 -- 172.21.15.94:0/2734073984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0550023890 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.080+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c19ee80 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.080+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c1a1b70 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.081+0000 7f0559600700 1 -- 172.21.15.94:0/2734073984 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f055001b440 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.081+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c0640a0 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.082+0000 7f0559600700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0544039700 0x7f054403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.082+0000 7f0559600700 1 -- 172.21.15.94:0/2734073984 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f055004e020 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.082+0000 7f055be00700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0544039700 0x7f054403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.083+0000 7f055be00700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0544039700 0x7f054403bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0548006fd0 tx=0x7f0548006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.088+0000 7f0559600700 1 -- 172.21.15.94:0/2734073984 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f055001f030 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.318+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f055c1a1d40 con 0x7f055c07cd90 2024-04-25T14:52:09.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.319+0000 7f0559600700 1 -- 172.21.15.94:0/2734073984 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f05500185b0 con 0x7f055c07cd90 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0544039700 msgr2=0x7f054403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0544039700 0x7f054403bbc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0548006fd0 tx=0x7f0548006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 msgr2=0x7f055c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c19e6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0550000c00 tx=0x7f0550011aa0 comp rx=0 tx=0).stop 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 shutdown_connections 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0544039700 0x7f054403bbc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 --2- 172.21.15.94:0/2734073984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f055c07cd90 0x7f055c19e6b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.322+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 >> 172.21.15.94:0/2734073984 conn(0x7f055c103be0 msgr2=0x7f055c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.323+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 shutdown_connections 2024-04-25T14:52:09.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:08.323+0000 7f0563c00700 1 -- 172.21.15.94:0/2734073984 wait complete. 2024-04-25T14:52:09.095 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T14:52:09.095 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-04-25T14:52:09.118 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T14:52:09.118 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-04-25T14:52:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-04-25T14:52:09.119 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T14:52:09.126 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-04-25T14:52:09.126 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:09.126 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-04-25T14:52:09.126 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-04-25T14:52:09.126 INFO:teuthology.orchestra.run.smithi094.stdout: Password: 6xwy13mkve 2024-04-25T14:52:09.127 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config directory 2024-04-25T14:52:09.129 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling autotune for osd_memory_target 2024-04-25T14:52:09.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:09 smithi094 ceph-mon[27862]: mgrmap e13: x(active, since 6s) 2024-04-25T14:52:09.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:09 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2734073984' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T14:52:11.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.088+0000 7f71efe00700 1 Processor -- start 2024-04-25T14:52:11.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.088+0000 7f71efe00700 1 -- start start 2024-04-25T14:52:11.068 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.089+0000 7f71efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e8108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:11.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.089+0000 7f71efe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71e8108fd0 con 0x7f71e81085e0 2024-04-25T14:52:11.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.089+0000 7f71ed400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e8108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:11.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.089+0000 7f71ed400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e8108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38728/0 (socket says 172.21.15.94:38728) 2024-04-25T14:52:11.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.089+0000 7f71ed400700 1 -- 172.21.15.94:0/2286490910 learned_addr learned my addr 172.21.15.94:0/2286490910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:11.069 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.090+0000 7f71ed400700 1 -- 172.21.15.94:0/2286490910 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e8109790 con 0x7f71e81085e0 2024-04-25T14:52:11.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.090+0000 7f71ed400700 1 --2- 172.21.15.94:0/2286490910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e8108a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0098a0 tx=0x7f71dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14fd5a9d369b2136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:11.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.090+0000 7f71e7e00700 1 -- 172.21.15.94:0/2286490910 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71dc004160 con 0x7f71e81085e0 2024-04-25T14:52:11.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.090+0000 7f71e7e00700 1 -- 172.21.15.94:0/2286490910 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71dc004300 con 0x7f71e81085e0 2024-04-25T14:52:11.070 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.090+0000 7f71e7e00700 1 -- 172.21.15.94:0/2286490910 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71dc011a20 con 0x7f71e81085e0 2024-04-25T14:52:11.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.091+0000 7f71efe00700 1 -- 172.21.15.94:0/2286490910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 msgr2=0x7f71e8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:11.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.091+0000 7f71efe00700 1 --2- 172.21.15.94:0/2286490910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e8108a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0098a0 tx=0x7f71dc009bb0 comp rx=0 tx=0).stop 2024-04-25T14:52:11.071 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.091+0000 7f71efe00700 1 -- 172.21.15.94:0/2286490910 shutdown_connections 2024-04-25T14:52:11.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.091+0000 7f71efe00700 1 --2- 172.21.15.94:0/2286490910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e8108a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:11.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.091+0000 7f71efe00700 1 -- 172.21.15.94:0/2286490910 >> 172.21.15.94:0/2286490910 conn(0x7f71e8103c60 msgr2=0x7f71e8106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:11.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.091+0000 7f71efe00700 1 -- 172.21.15.94:0/2286490910 shutdown_connections 2024-04-25T14:52:11.072 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.091+0000 7f71efe00700 1 -- 172.21.15.94:0/2286490910 wait complete. 2024-04-25T14:52:11.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.092+0000 7f71efe00700 1 Processor -- start 2024-04-25T14:52:11.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.092+0000 7f71efe00700 1 -- start start 2024-04-25T14:52:11.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.093+0000 7f71efe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:11.073 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.093+0000 7f71efe00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810cb40 con 0x7f71e81085e0 2024-04-25T14:52:11.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.093+0000 7f71ed400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:11.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.093+0000 7f71ed400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38744/0 (socket says 172.21.15.94:38744) 2024-04-25T14:52:11.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.093+0000 7f71ed400700 1 -- 172.21.15.94:0/1349258052 learned_addr learned my addr 172.21.15.94:0/1349258052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:11.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.093+0000 7f71ed400700 1 -- 172.21.15.94:0/1349258052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e819ec10 con 0x7f71e81085e0 2024-04-25T14:52:11.074 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.094+0000 7f71ed400700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e819e6d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71dc000c00 tx=0x7f71dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:11.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.094+0000 7f71e6000700 1 -- 172.21.15.94:0/1349258052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71dc003aa0 con 0x7f71e81085e0 2024-04-25T14:52:11.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.094+0000 7f71e6000700 1 -- 172.21.15.94:0/1349258052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71dc01b8f0 con 0x7f71e81085e0 2024-04-25T14:52:11.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.094+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e819eea0 con 0x7f71e81085e0 2024-04-25T14:52:11.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.094+0000 7f71e6000700 1 -- 172.21.15.94:0/1349258052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71dc023890 con 0x7f71e81085e0 2024-04-25T14:52:11.075 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.094+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e81a1b90 con 0x7f71e81085e0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.095+0000 7f71e6000700 1 -- 172.21.15.94:0/1349258052 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f71dc01b440 con 0x7f71e81085e0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.095+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e810ccc0 con 0x7f71e81085e0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.096+0000 7f71e6000700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f71d0039700 0x7f71d003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.096+0000 7f71e6000700 1 -- 172.21.15.94:0/1349258052 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f71dc04e0b0 con 0x7f71e81085e0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.096+0000 7f71eca00700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f71d0039700 0x7f71d003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.097+0000 7f71eca00700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f71d0039700 0x7f71d003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71d4006fd0 tx=0x7f71d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.101+0000 7f71e6000700 1 -- 172.21.15.94:0/1349258052 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f71dc01f030 con 0x7f71e81085e0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.326+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f71e81a1d60 con 0x7f71e81085e0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.327+0000 7f71e6000700 1 -- 172.21.15.94:0/1349258052 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f71dc016260 con 0x7f71e81085e0 2024-04-25T14:52:11.078 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.330+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f71d0039700 msgr2=0x7f71d003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.330+0000 7f71efe00700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f71d0039700 0x7f71d003bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71d4006fd0 tx=0x7f71d4006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.330+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 msgr2=0x7f71e819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.330+0000 7f71efe00700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e819e6d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71dc000c00 tx=0x7f71dc011aa0 comp rx=0 tx=0).stop 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.331+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 shutdown_connections 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.331+0000 7f71efe00700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f71d0039700 0x7f71d003bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.331+0000 7f71efe00700 1 --2- 172.21.15.94:0/1349258052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71e81085e0 0x7f71e819e6d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.331+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 >> 172.21.15.94:0/1349258052 conn(0x7f71e8103c60 msgr2=0x7f71e8104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.331+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 shutdown_connections 2024-04-25T14:52:11.079 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:10.331+0000 7f71efe00700 1 -- 172.21.15.94:0/1349258052 wait complete. 2024-04-25T14:52:12.637 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.676+0000 7f5642800700 1 Processor -- start 2024-04-25T14:52:12.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.677+0000 7f5642800700 1 -- start start 2024-04-25T14:52:12.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.677+0000 7f5642800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c10ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:12.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.677+0000 7f5642800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f563c10b1f0 con 0x7f563c10a800 2024-04-25T14:52:12.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.678+0000 7f563b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c10ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:12.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.678+0000 7f563b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c10ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38756/0 (socket says 172.21.15.94:38756) 2024-04-25T14:52:12.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.678+0000 7f563b400700 1 -- 172.21.15.94:0/158105923 learned_addr learned my addr 172.21.15.94:0/158105923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:12.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.678+0000 7f563b400700 1 -- 172.21.15.94:0/158105923 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c10b9b0 con 0x7f563c10a800 2024-04-25T14:52:12.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.678+0000 7f563b400700 1 --2- 172.21.15.94:0/158105923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c10ac20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f562c009e80 tx=0x7f562c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930333d3a27497fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:12.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.679+0000 7f563aa00700 1 -- 172.21.15.94:0/158105923 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f562c017070 con 0x7f563c10a800 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.679+0000 7f563aa00700 1 -- 172.21.15.94:0/158105923 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c00b8c0 con 0x7f563c10a800 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.679+0000 7f563aa00700 1 -- 172.21.15.94:0/158105923 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f562c004490 con 0x7f563c10a800 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.680+0000 7f5642800700 1 -- 172.21.15.94:0/158105923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 msgr2=0x7f563c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.680+0000 7f5642800700 1 --2- 172.21.15.94:0/158105923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c10ac20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f562c009e80 tx=0x7f562c00b330 comp rx=0 tx=0).stop 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.680+0000 7f5642800700 1 -- 172.21.15.94:0/158105923 shutdown_connections 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.680+0000 7f5642800700 1 --2- 172.21.15.94:0/158105923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c10ac20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.680+0000 7f5642800700 1 -- 172.21.15.94:0/158105923 >> 172.21.15.94:0/158105923 conn(0x7f563c07dbd0 msgr2=0x7f563c1084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.680+0000 7f5642800700 1 -- 172.21.15.94:0/158105923 shutdown_connections 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.680+0000 7f5642800700 1 -- 172.21.15.94:0/158105923 wait complete. 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.681+0000 7f5642800700 1 Processor -- start 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.681+0000 7f5642800700 1 -- start start 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.681+0000 7f5642800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.681+0000 7f5642800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f563c10ed60 con 0x7f563c10a800 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.682+0000 7f563b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:12.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.682+0000 7f563b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38758/0 (socket says 172.21.15.94:38758) 2024-04-25T14:52:12.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.682+0000 7f563b400700 1 -- 172.21.15.94:0/667617460 learned_addr learned my addr 172.21.15.94:0/667617460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:12.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.682+0000 7f563b400700 1 -- 172.21.15.94:0/667617460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c19ebf0 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.682+0000 7f563b400700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c19e6b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f562c00bca0 tx=0x7f562c00bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.682+0000 7f5638c00700 1 -- 172.21.15.94:0/667617460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f562c020020 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.682+0000 7f5638c00700 1 -- 172.21.15.94:0/667617460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c003b50 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.683+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c19ee80 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.683+0000 7f5638c00700 1 -- 172.21.15.94:0/667617460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f562c017040 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.683+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c1a1b70 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.684+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c10eee0 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.684+0000 7f5638c00700 1 -- 172.21.15.94:0/667617460 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f562c023020 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.684+0000 7f5638c00700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f56200396f0 0x7f562003bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.684+0000 7f5638c00700 1 -- 172.21.15.94:0/667617460 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f562c00f050 con 0x7f563c10a800 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.685+0000 7f5632a00700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f56200396f0 0x7f562003bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.685+0000 7f5632a00700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f56200396f0 0x7f562003bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5624006fd0 tx=0x7f5624006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:12.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:11.690+0000 7f5638c00700 1 -- 172.21.15.94:0/667617460 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f562c02a020 con 0x7f563c10a800 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.033+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f563c1a1d40 con 0x7f563c10a800 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.038+0000 7f5638c00700 1 -- 172.21.15.94:0/667617460 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f562c02b260 con 0x7f563c10a800 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.041+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f56200396f0 msgr2=0x7f562003bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.041+0000 7f5642800700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f56200396f0 0x7f562003bbb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5624006fd0 tx=0x7f5624006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.041+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 msgr2=0x7f563c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.041+0000 7f5642800700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c19e6b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f562c00bca0 tx=0x7f562c00bd80 comp rx=0 tx=0).stop 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.042+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 shutdown_connections 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.042+0000 7f5642800700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f56200396f0 0x7f562003bbb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.042+0000 7f5642800700 1 --2- 172.21.15.94:0/667617460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f563c10a800 0x7f563c19e6b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.042+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 >> 172.21.15.94:0/667617460 conn(0x7f563c07dbd0 msgr2=0x7f563c107c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.042+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 shutdown_connections 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-04-25T14:52:12.042+0000 7f5642800700 1 -- 172.21.15.94:0/667617460 wait complete. 2024-04-25T14:52:12.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T14:52:12.647 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T14:52:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.647 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T14:52:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.647 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:12.648 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-04-25T14:52:12.735 INFO:tasks.cephadm:Fetching config... 2024-04-25T14:52:12.736 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:52:12.736 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T14:52:12.784 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T14:52:12.784 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:52:12.784 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T14:52:12.865 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T14:52:12.865 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:52:12.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T14:52:12.985 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T14:52:12.985 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:52:12.985 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T14:52:13.034 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T14:52:13.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7bxQUxxNc1aQJNgUe/jMN3nmsh+0+kibcaPde8lWQqxZAJZ5ggU7ef9la+LvPLBwt33rJQGQUQj7R8qkDCPSsV0QRqIBnCUJXfnFfWpbQR3dsH+fiSeQM5CO+xEe7TE/FtQOIi6DAP/8kKb+rTin9iUMiDRRPsaAwFDzdC5dA0o4ua32A4m+E2bYKJxe5uYN35VbTQN8IqwgeITIB8P7lCqR+rvmoVy1V62TC11xfqtPuPGd9TIhphnahF2PI4WTLaabHWrJonow3J/+ZJdrUKNdi3s+fEBN0zIZAE7KBCxy/OCtXxC+jKMXs6cOKWlE0nkExPxHT9yxocXO3ZKyGiw1rsGcHEl/93qlO0zu+ibiCQjybuOhloc7Gy3W3QEj6K5ig47w/xQjzRBpJi1PusKFbjff8n7Mf4UT3LAV0IO7UkYpE44b5Y1+uzpoHI6gLBwPl9IJmKAk2YokZlygT0QdmeKDe848NY2PUiM75ml/iZMqI22zoNxAqGAiUrDU= ceph-f04d929c-0312-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:52:13.180 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:13 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/667617460' entity='client.admin' 2024-04-25T14:52:13.187 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7bxQUxxNc1aQJNgUe/jMN3nmsh+0+kibcaPde8lWQqxZAJZ5ggU7ef9la+LvPLBwt33rJQGQUQj7R8qkDCPSsV0QRqIBnCUJXfnFfWpbQR3dsH+fiSeQM5CO+xEe7TE/FtQOIi6DAP/8kKb+rTin9iUMiDRRPsaAwFDzdC5dA0o4ua32A4m+E2bYKJxe5uYN35VbTQN8IqwgeITIB8P7lCqR+rvmoVy1V62TC11xfqtPuPGd9TIhphnahF2PI4WTLaabHWrJonow3J/+ZJdrUKNdi3s+fEBN0zIZAE7KBCxy/OCtXxC+jKMXs6cOKWlE0nkExPxHT9yxocXO3ZKyGiw1rsGcHEl/93qlO0zu+ibiCQjybuOhloc7Gy3W3QEj6K5ig47w/xQjzRBpJi1PusKFbjff8n7Mf4UT3LAV0IO7UkYpE44b5Y1+uzpoHI6gLBwPl9IJmKAk2YokZlygT0QdmeKDe848NY2PUiM75ml/iZMqI22zoNxAqGAiUrDU= ceph-f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:52:13.221 DEBUG:teuthology.orchestra.run.smithi137:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7bxQUxxNc1aQJNgUe/jMN3nmsh+0+kibcaPde8lWQqxZAJZ5ggU7ef9la+LvPLBwt33rJQGQUQj7R8qkDCPSsV0QRqIBnCUJXfnFfWpbQR3dsH+fiSeQM5CO+xEe7TE/FtQOIi6DAP/8kKb+rTin9iUMiDRRPsaAwFDzdC5dA0o4ua32A4m+E2bYKJxe5uYN35VbTQN8IqwgeITIB8P7lCqR+rvmoVy1V62TC11xfqtPuPGd9TIhphnahF2PI4WTLaabHWrJonow3J/+ZJdrUKNdi3s+fEBN0zIZAE7KBCxy/OCtXxC+jKMXs6cOKWlE0nkExPxHT9yxocXO3ZKyGiw1rsGcHEl/93qlO0zu+ibiCQjybuOhloc7Gy3W3QEj6K5ig47w/xQjzRBpJi1PusKFbjff8n7Mf4UT3LAV0IO7UkYpE44b5Y1+uzpoHI6gLBwPl9IJmKAk2YokZlygT0QdmeKDe848NY2PUiM75ml/iZMqI22zoNxAqGAiUrDU= ceph-f04d929c-0312-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:52:13.262 INFO:teuthology.orchestra.run.smithi137.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7bxQUxxNc1aQJNgUe/jMN3nmsh+0+kibcaPde8lWQqxZAJZ5ggU7ef9la+LvPLBwt33rJQGQUQj7R8qkDCPSsV0QRqIBnCUJXfnFfWpbQR3dsH+fiSeQM5CO+xEe7TE/FtQOIi6DAP/8kKb+rTin9iUMiDRRPsaAwFDzdC5dA0o4ua32A4m+E2bYKJxe5uYN35VbTQN8IqwgeITIB8P7lCqR+rvmoVy1V62TC11xfqtPuPGd9TIhphnahF2PI4WTLaabHWrJonow3J/+ZJdrUKNdi3s+fEBN0zIZAE7KBCxy/OCtXxC+jKMXs6cOKWlE0nkExPxHT9yxocXO3ZKyGiw1rsGcHEl/93qlO0zu+ibiCQjybuOhloc7Gy3W3QEj6K5ig47w/xQjzRBpJi1PusKFbjff8n7Mf4UT3LAV0IO7UkYpE44b5Y1+uzpoHI6gLBwPl9IJmKAk2YokZlygT0QdmeKDe848NY2PUiM75ml/iZMqI22zoNxAqGAiUrDU= ceph-f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:52:13.274 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7bxQUxxNc1aQJNgUe/jMN3nmsh+0+kibcaPde8lWQqxZAJZ5ggU7ef9la+LvPLBwt33rJQGQUQj7R8qkDCPSsV0QRqIBnCUJXfnFfWpbQR3dsH+fiSeQM5CO+xEe7TE/FtQOIi6DAP/8kKb+rTin9iUMiDRRPsaAwFDzdC5dA0o4ua32A4m+E2bYKJxe5uYN35VbTQN8IqwgeITIB8P7lCqR+rvmoVy1V62TC11xfqtPuPGd9TIhphnahF2PI4WTLaabHWrJonow3J/+ZJdrUKNdi3s+fEBN0zIZAE7KBCxy/OCtXxC+jKMXs6cOKWlE0nkExPxHT9yxocXO3ZKyGiw1rsGcHEl/93qlO0zu+ibiCQjybuOhloc7Gy3W3QEj6K5ig47w/xQjzRBpJi1PusKFbjff8n7Mf4UT3LAV0IO7UkYpE44b5Y1+uzpoHI6gLBwPl9IJmKAk2YokZlygT0QdmeKDe848NY2PUiM75ml/iZMqI22zoNxAqGAiUrDU= ceph-f04d929c-0312-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T14:52:13.316 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC7bxQUxxNc1aQJNgUe/jMN3nmsh+0+kibcaPde8lWQqxZAJZ5ggU7ef9la+LvPLBwt33rJQGQUQj7R8qkDCPSsV0QRqIBnCUJXfnFfWpbQR3dsH+fiSeQM5CO+xEe7TE/FtQOIi6DAP/8kKb+rTin9iUMiDRRPsaAwFDzdC5dA0o4ua32A4m+E2bYKJxe5uYN35VbTQN8IqwgeITIB8P7lCqR+rvmoVy1V62TC11xfqtPuPGd9TIhphnahF2PI4WTLaabHWrJonow3J/+ZJdrUKNdi3s+fEBN0zIZAE7KBCxy/OCtXxC+jKMXs6cOKWlE0nkExPxHT9yxocXO3ZKyGiw1rsGcHEl/93qlO0zu+ibiCQjybuOhloc7Gy3W3QEj6K5ig47w/xQjzRBpJi1PusKFbjff8n7Mf4UT3LAV0IO7UkYpE44b5Y1+uzpoHI6gLBwPl9IJmKAk2YokZlygT0QdmeKDe848NY2PUiM75ml/iZMqI22zoNxAqGAiUrDU= ceph-f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:52:13.328 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T14:52:13.738 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:52:16.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.956+0000 7f3e63200700 1 -- 172.21.15.94:0/3337061528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 msgr2=0x7f3e5c074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:16.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.956+0000 7f3e63200700 1 --2- 172.21.15.94:0/3337061528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c074ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00b0a0 tx=0x7f3e4c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:52:16.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.956+0000 7f3e63200700 1 -- 172.21.15.94:0/3337061528 shutdown_connections 2024-04-25T14:52:16.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.956+0000 7f3e63200700 1 --2- 172.21.15.94:0/3337061528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c074ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:16.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.956+0000 7f3e63200700 1 -- 172.21.15.94:0/3337061528 >> 172.21.15.94:0/3337061528 conn(0x7f3e5c0fda40 msgr2=0x7f3e5c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.956+0000 7f3e63200700 1 -- 172.21.15.94:0/3337061528 shutdown_connections 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.956+0000 7f3e63200700 1 -- 172.21.15.94:0/3337061528 wait complete. 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.957+0000 7f3e63200700 1 Processor -- start 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.957+0000 7f3e63200700 1 -- start start 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.958+0000 7f3e63200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c19e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.958+0000 7f3e63200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5c10ea80 con 0x7f3e5c076a70 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.958+0000 7f3e5be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c19e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.958+0000 7f3e5be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c19e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38780/0 (socket says 172.21.15.94:38780) 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.958+0000 7f3e5be00700 1 -- 172.21.15.94:0/1679920480 learned_addr learned my addr 172.21.15.94:0/1679920480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.959+0000 7f3e5be00700 1 -- 172.21.15.94:0/1679920480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5c19e570 con 0x7f3e5c076a70 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.959+0000 7f3e5be00700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c19e030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00d010 tx=0x7f3e4c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.959+0000 7f3e59600700 1 -- 172.21.15.94:0/1679920480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e4c021020 con 0x7f3e5c076a70 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.960+0000 7f3e59600700 1 -- 172.21.15.94:0/1679920480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e4c00bdd0 con 0x7f3e5c076a70 2024-04-25T14:52:16.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.960+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e5c19e800 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.960+0000 7f3e59600700 1 -- 172.21.15.94:0/1679920480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e4c017040 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.960+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5c1a14f0 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.961+0000 7f3e59600700 1 -- 172.21.15.94:0/1679920480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3e4c024020 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.961+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e5c10f160 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.961+0000 7f3e59600700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e400396b0 0x7f3e4003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.962+0000 7f3e59600700 1 -- 172.21.15.94:0/1679920480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e4c00f050 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.962+0000 7f3e53400700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e400396b0 0x7f3e4003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.962+0000 7f3e53400700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e400396b0 0x7f3e4003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006fd0 tx=0x7f3e44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:15.967+0000 7f3e59600700 1 -- 172.21.15.94:0/1679920480 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e4c02b020 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.198+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3e5c02dff0 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.201+0000 7f3e59600700 1 -- 172.21.15.94:0/1679920480 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3e4c0253b0 con 0x7f3e5c076a70 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e400396b0 msgr2=0x7f3e4003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e400396b0 0x7f3e4003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3e44006fd0 tx=0x7f3e44006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 msgr2=0x7f3e5c19e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c19e030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00d010 tx=0x7f3e4c003ce0 comp rx=0 tx=0).stop 2024-04-25T14:52:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 shutdown_connections 2024-04-25T14:52:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e400396b0 0x7f3e4003bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 --2- 172.21.15.94:0/1679920480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3e5c076a70 0x7f3e5c19e030 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 >> 172.21.15.94:0/1679920480 conn(0x7f3e5c0fda40 msgr2=0x7f3e5c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 shutdown_connections 2024-04-25T14:52:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:16.205+0000 7f3e63200700 1 -- 172.21.15.94:0/1679920480 wait complete. 2024-04-25T14:52:16.871 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T14:52:16.872 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T14:52:17.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:17 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1679920480' entity='client.admin' 2024-04-25T14:52:17.470 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:52:19.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.847+0000 7fad4be00700 1 -- 172.21.15.94:0/2053657026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 msgr2=0x7fad441028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.847+0000 7fad4be00700 1 --2- 172.21.15.94:0/2053657026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad441028c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fad38009910 tx=0x7fad38009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.847+0000 7fad4be00700 1 -- 172.21.15.94:0/2053657026 shutdown_connections 2024-04-25T14:52:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.847+0000 7fad4be00700 1 --2- 172.21.15.94:0/2053657026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad441028c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.847+0000 7fad4be00700 1 -- 172.21.15.94:0/2053657026 >> 172.21.15.94:0/2053657026 conn(0x7fad440fda40 msgr2=0x7fad440ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.847+0000 7fad4be00700 1 -- 172.21.15.94:0/2053657026 shutdown_connections 2024-04-25T14:52:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.847+0000 7fad4be00700 1 -- 172.21.15.94:0/2053657026 wait complete. 2024-04-25T14:52:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.849+0000 7fad4be00700 1 Processor -- start 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.849+0000 7fad4be00700 1 -- start start 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.849+0000 7fad4be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad44076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.849+0000 7fad4be00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad441083e0 con 0x7fad441024a0 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.850+0000 7fad49400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad44076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.850+0000 7fad49400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad44076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42474/0 (socket says 172.21.15.94:42474) 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.850+0000 7fad49400700 1 -- 172.21.15.94:0/1298670813 learned_addr learned my addr 172.21.15.94:0/1298670813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.850+0000 7fad49400700 1 -- 172.21.15.94:0/1298670813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad44076c40 con 0x7fad441024a0 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.850+0000 7fad49400700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad44076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad38005950 tx=0x7fad38004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.850+0000 7fad42000700 1 -- 172.21.15.94:0/1298670813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad38026070 con 0x7fad441024a0 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.851+0000 7fad42000700 1 -- 172.21.15.94:0/1298670813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad3800bc00 con 0x7fad441024a0 2024-04-25T14:52:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.851+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad44074e30 con 0x7fad441024a0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.851+0000 7fad42000700 1 -- 172.21.15.94:0/1298670813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad38003a10 con 0x7fad441024a0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.851+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad44075370 con 0x7fad441024a0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.852+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad44108b60 con 0x7fad441024a0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.852+0000 7fad42000700 1 -- 172.21.15.94:0/1298670813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fad38029070 con 0x7fad441024a0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.852+0000 7fad42000700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad2c0396b0 0x7fad2c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.852+0000 7fad42000700 1 -- 172.21.15.94:0/1298670813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fad3801e070 con 0x7fad441024a0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.853+0000 7fad48a00700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad2c0396b0 0x7fad2c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.853+0000 7fad48a00700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad2c0396b0 0x7fad2c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad30006fd0 tx=0x7fad30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:19.858+0000 7fad42000700 1 -- 172.21.15.94:0/1298670813 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad38013070 con 0x7fad441024a0 2024-04-25T14:52:20.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.098+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fad44190e60 con 0x7fad2c0396b0 2024-04-25T14:52:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.103+0000 7fad42000700 1 -- 172.21.15.94:0/1298670813 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fad44190e60 con 0x7fad2c0396b0 2024-04-25T14:52:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad2c0396b0 msgr2=0x7fad2c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad2c0396b0 0x7fad2c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fad30006fd0 tx=0x7fad30006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 msgr2=0x7fad44076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad44076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fad38005950 tx=0x7fad38004fa0 comp rx=0 tx=0).stop 2024-04-25T14:52:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 shutdown_connections 2024-04-25T14:52:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad2c0396b0 0x7fad2c03bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 --2- 172.21.15.94:0/1298670813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad441024a0 0x7fad44076700 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 >> 172.21.15.94:0/1298670813 conn(0x7fad440fda40 msgr2=0x7fad440fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 shutdown_connections 2024-04-25T14:52:20.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:20.106+0000 7fad4be00700 1 -- 172.21.15.94:0/1298670813 wait complete. 2024-04-25T14:52:20.848 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi137 2024-04-25T14:52:20.848 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:52:20.848 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:52:20.867 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:52:20.867 DEBUG:teuthology.orchestra.run.smithi137:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:20.924 INFO:tasks.cephadm:Adding host smithi137 to orchestrator... 2024-04-25T14:52:20.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch host add smithi137 2024-04-25T14:52:21.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:52:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:20 smithi094 ceph-mon[27862]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T14:52:21.326 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:52:22.188 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:21 smithi094 ceph-mon[27862]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:52:22.189 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:21 smithi094 ceph-mon[27862]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:22.189 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:21 smithi094 ceph-mon[27862]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:23.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:22 smithi094 ceph-mon[27862]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:52:23.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:23.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:23.072 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:23.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.241+0000 7ffa3f800700 1 -- 172.21.15.94:0/459902857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 msgr2=0x7ffa3810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:23.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.241+0000 7ffa3f800700 1 --2- 172.21.15.94:0/459902857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3810dd40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009910 tx=0x7ffa2c009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:23.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.241+0000 7ffa3f800700 1 -- 172.21.15.94:0/459902857 shutdown_connections 2024-04-25T14:52:23.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.241+0000 7ffa3f800700 1 --2- 172.21.15.94:0/459902857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3810dd40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:23.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.241+0000 7ffa3f800700 1 -- 172.21.15.94:0/459902857 >> 172.21.15.94:0/459902857 conn(0x7ffa380781e0 msgr2=0x7ffa3807a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:23.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.241+0000 7ffa3f800700 1 -- 172.21.15.94:0/459902857 shutdown_connections 2024-04-25T14:52:23.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.241+0000 7ffa3f800700 1 -- 172.21.15.94:0/459902857 wait complete. 2024-04-25T14:52:23.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.242+0000 7ffa3f800700 1 Processor -- start 2024-04-25T14:52:23.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.243+0000 7ffa3f800700 1 -- start start 2024-04-25T14:52:23.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.243+0000 7ffa3f800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:23.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.243+0000 7ffa3f800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffa38112f60 con 0x7ffa3810d960 2024-04-25T14:52:23.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.243+0000 7ffa3ce00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:23.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.243+0000 7ffa3ce00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3819e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42494/0 (socket says 172.21.15.94:42494) 2024-04-25T14:52:23.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.243+0000 7ffa3ce00700 1 -- 172.21.15.94:0/3621293031 learned_addr learned my addr 172.21.15.94:0/3621293031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:23.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.244+0000 7ffa3ce00700 1 -- 172.21.15.94:0/3621293031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3819ea40 con 0x7ffa3810d960 2024-04-25T14:52:23.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.244+0000 7ffa3ce00700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3819e500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c006010 tx=0x7ffa2c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:23.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.244+0000 7ffa35600700 1 -- 172.21.15.94:0/3621293031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa2c026070 con 0x7ffa3810d960 2024-04-25T14:52:23.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.244+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3819ed30 con 0x7ffa3810d960 2024-04-25T14:52:23.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.244+0000 7ffa35600700 1 -- 172.21.15.94:0/3621293031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2c00bba0 con 0x7ffa3810d960 2024-04-25T14:52:23.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.244+0000 7ffa35600700 1 -- 172.21.15.94:0/3621293031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa2c003930 con 0x7ffa3810d960 2024-04-25T14:52:23.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.245+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa381a2dd0 con 0x7ffa3810d960 2024-04-25T14:52:23.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.246+0000 7ffa35600700 1 -- 172.21.15.94:0/3621293031 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ffa2c029070 con 0x7ffa3810d960 2024-04-25T14:52:23.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.246+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3810a180 con 0x7ffa3810d960 2024-04-25T14:52:23.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.246+0000 7ffa35600700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffa200396b0 0x7ffa2003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:23.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.246+0000 7ffa35600700 1 -- 172.21.15.94:0/3621293031 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffa2c01e070 con 0x7ffa3810d960 2024-04-25T14:52:23.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.246+0000 7ffa37e00700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffa200396b0 0x7ffa2003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:23.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.247+0000 7ffa37e00700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffa200396b0 0x7ffa2003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006fd0 tx=0x7ffa24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:23.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.252+0000 7ffa35600700 1 -- 172.21.15.94:0/3621293031 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa2c013070 con 0x7ffa3810d960 2024-04-25T14:52:23.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:23.482+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}) -- 0x7ffa3819f860 con 0x7ffa200396b0 2024-04-25T14:52:24.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:23 smithi094 ceph-mon[27862]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:24.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:23 smithi094 ceph-mon[27862]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi137", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:25.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:24 smithi094 ceph-mon[27862]: Deploying cephadm binary to smithi137 2024-04-25T14:52:26.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:25 smithi094 ceph-mon[27862]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:28.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.151+0000 7ffa35600700 1 -- 172.21.15.94:0/3621293031 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ffa3819f860 con 0x7ffa200396b0 2024-04-25T14:52:28.153 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi137' with addr '172.21.15.137' 2024-04-25T14:52:28.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.154+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffa200396b0 msgr2=0x7ffa2003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:28.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.154+0000 7ffa3f800700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffa200396b0 0x7ffa2003bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffa24006fd0 tx=0x7ffa24006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:28.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.154+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 msgr2=0x7ffa3819e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:28.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.154+0000 7ffa3f800700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3819e500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c006010 tx=0x7ffa2c005780 comp rx=0 tx=0).stop 2024-04-25T14:52:28.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.155+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 shutdown_connections 2024-04-25T14:52:28.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.155+0000 7ffa3f800700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffa200396b0 0x7ffa2003bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:28.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.155+0000 7ffa3f800700 1 --2- 172.21.15.94:0/3621293031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa3810d960 0x7ffa3819e500 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:28.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.155+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 >> 172.21.15.94:0/3621293031 conn(0x7ffa380781e0 msgr2=0x7ffa38110580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:28.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.155+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 shutdown_connections 2024-04-25T14:52:28.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:28.155+0000 7ffa3f800700 1 -- 172.21.15.94:0/3621293031 wait complete. 2024-04-25T14:52:28.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:28 smithi094 ceph-mon[27862]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:28.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:28.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:28.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:52:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:52:29.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:29 smithi094 ceph-mon[27862]: Added host smithi137 2024-04-25T14:52:29.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:29.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:30.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:30 smithi094 ceph-mon[27862]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.257+0000 7f106a200700 1 -- 172.21.15.94:0/2291115677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 msgr2=0x7f10640751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.257+0000 7f106a200700 1 --2- 172.21.15.94:0/2291115677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f10640751c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1054009910 tx=0x7f1054009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.257+0000 7f106a200700 1 -- 172.21.15.94:0/2291115677 shutdown_connections 2024-04-25T14:52:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.257+0000 7f106a200700 1 --2- 172.21.15.94:0/2291115677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f10640751c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.257+0000 7f106a200700 1 -- 172.21.15.94:0/2291115677 >> 172.21.15.94:0/2291115677 conn(0x7f10640fdf80 msgr2=0x7f10641003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.258+0000 7f106a200700 1 -- 172.21.15.94:0/2291115677 shutdown_connections 2024-04-25T14:52:31.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.258+0000 7f106a200700 1 -- 172.21.15.94:0/2291115677 wait complete. 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.259+0000 7f106a200700 1 Processor -- start 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.259+0000 7f106a200700 1 -- start start 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.259+0000 7f106a200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f106419e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.259+0000 7f106a200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1064112fa0 con 0x7f1064074de0 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.259+0000 7f1063400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f106419e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.259+0000 7f1063400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f106419e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54756/0 (socket says 172.21.15.94:54756) 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.259+0000 7f1063400700 1 -- 172.21.15.94:0/4233971992 learned_addr learned my addr 172.21.15.94:0/4233971992 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.260+0000 7f1063400700 1 -- 172.21.15.94:0/4233971992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106419ea80 con 0x7f1064074de0 2024-04-25T14:52:31.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.260+0000 7f1063400700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f106419e540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1054006010 tx=0x7f1054005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.260+0000 7f1060c00700 1 -- 172.21.15.94:0/4233971992 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054026070 con 0x7f1064074de0 2024-04-25T14:52:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.260+0000 7f1060c00700 1 -- 172.21.15.94:0/4233971992 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f105400bc00 con 0x7f1064074de0 2024-04-25T14:52:31.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.260+0000 7f1060c00700 1 -- 172.21.15.94:0/4233971992 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054003910 con 0x7f1064074de0 2024-04-25T14:52:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.260+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106419ed10 con 0x7f1064074de0 2024-04-25T14:52:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.260+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106419f170 con 0x7f1064074de0 2024-04-25T14:52:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.261+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1064076520 con 0x7f1064074de0 2024-04-25T14:52:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.262+0000 7f1060c00700 1 -- 172.21.15.94:0/4233971992 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1054029070 con 0x7f1064074de0 2024-04-25T14:52:31.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.262+0000 7f1060c00700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f10480396b0 0x7f104803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.262+0000 7f1060c00700 1 -- 172.21.15.94:0/4233971992 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f105401e070 con 0x7f1064074de0 2024-04-25T14:52:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.262+0000 7f105aa00700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f10480396b0 0x7f104803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:31.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.263+0000 7f105aa00700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f10480396b0 0x7f104803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:31.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.268+0000 7f1060c00700 1 -- 172.21.15.94:0/4233971992 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1054013070 con 0x7f1064074de0 2024-04-25T14:52:31.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.501+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f106419f8a0 con 0x7f10480396b0 2024-04-25T14:52:31.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.504+0000 7f1060c00700 1 -- 172.21.15.94:0/4233971992 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f106419f8a0 con 0x7f10480396b0 2024-04-25T14:52:31.504 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:31.504 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}] 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.506+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f10480396b0 msgr2=0x7f104803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.506+0000 7f106a200700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f10480396b0 0x7f104803bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f104c006fd0 tx=0x7f104c006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.506+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 msgr2=0x7f106419e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.506+0000 7f106a200700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f106419e540 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1054006010 tx=0x7f1054005780 comp rx=0 tx=0).stop 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.507+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 shutdown_connections 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.507+0000 7f106a200700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f10480396b0 0x7f104803bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.507+0000 7f106a200700 1 --2- 172.21.15.94:0/4233971992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1064074de0 0x7f106419e540 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:31.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.507+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 >> 172.21.15.94:0/4233971992 conn(0x7f10640fdf80 msgr2=0x7f10641041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.507+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 shutdown_connections 2024-04-25T14:52:31.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:31.507+0000 7f106a200700 1 -- 172.21.15.94:0/4233971992 wait complete. 2024-04-25T14:52:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:31 smithi094 ceph-mon[27862]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:31 smithi094 ceph-mon[27862]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:52:32.299 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-04-25T14:52:32.300 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:52:32.300 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-04-25T14:52:32.319 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:52:32.319 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:32.377 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-04-25T14:52:32.377 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch host add smithi138 2024-04-25T14:52:32.804 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:52:34.613 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:34.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:34.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:34.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:34.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:34.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:52:34.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:34.614 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:52:34.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.780+0000 7feb5e200700 1 -- 172.21.15.94:0/3459858642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 msgr2=0x7feb581088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:34.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.780+0000 7feb5e200700 1 --2- 172.21.15.94:0/3459858642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb581088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb48009910 tx=0x7feb48009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:34.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.780+0000 7feb5e200700 1 -- 172.21.15.94:0/3459858642 shutdown_connections 2024-04-25T14:52:34.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.780+0000 7feb5e200700 1 --2- 172.21.15.94:0/3459858642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb581088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:34.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.780+0000 7feb5e200700 1 -- 172.21.15.94:0/3459858642 >> 172.21.15.94:0/3459858642 conn(0x7feb580fdf80 msgr2=0x7feb581003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:34.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.780+0000 7feb5e200700 1 -- 172.21.15.94:0/3459858642 shutdown_connections 2024-04-25T14:52:34.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.780+0000 7feb5e200700 1 -- 172.21.15.94:0/3459858642 wait complete. 2024-04-25T14:52:34.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.782+0000 7feb5e200700 1 Processor -- start 2024-04-25T14:52:34.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.782+0000 7feb5e200700 1 -- start start 2024-04-25T14:52:34.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.782+0000 7feb5e200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb58197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:34.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.782+0000 7feb5e200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7feb5810c940 con 0x7feb581084e0 2024-04-25T14:52:34.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.783+0000 7feb57400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb58197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:34.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.783+0000 7feb57400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb58197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54770/0 (socket says 172.21.15.94:54770) 2024-04-25T14:52:34.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.783+0000 7feb57400700 1 -- 172.21.15.94:0/867348139 learned_addr learned my addr 172.21.15.94:0/867348139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:34.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.783+0000 7feb57400700 1 -- 172.21.15.94:0/867348139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb581983f0 con 0x7feb581084e0 2024-04-25T14:52:34.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.783+0000 7feb57400700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb58197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb480056d0 tx=0x7feb480057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:34.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.784+0000 7feb4fe00700 1 -- 172.21.15.94:0/867348139 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb48026070 con 0x7feb581084e0 2024-04-25T14:52:34.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.784+0000 7feb4fe00700 1 -- 172.21.15.94:0/867348139 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4800bba0 con 0x7feb581084e0 2024-04-25T14:52:34.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.784+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb581986e0 con 0x7feb581084e0 2024-04-25T14:52:34.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.784+0000 7feb4fe00700 1 -- 172.21.15.94:0/867348139 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb48003920 con 0x7feb581084e0 2024-04-25T14:52:34.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.784+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb5819c780 con 0x7feb581084e0 2024-04-25T14:52:34.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.786+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb38005330 con 0x7feb581084e0 2024-04-25T14:52:34.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.786+0000 7feb4fe00700 1 -- 172.21.15.94:0/867348139 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feb48029070 con 0x7feb581084e0 2024-04-25T14:52:34.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.786+0000 7feb4fe00700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb3c0396b0 0x7feb3c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:34.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.786+0000 7feb4fe00700 1 -- 172.21.15.94:0/867348139 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feb4801e070 con 0x7feb581084e0 2024-04-25T14:52:34.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.787+0000 7feb56a00700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb3c0396b0 0x7feb3c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:34.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.787+0000 7feb56a00700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb3c0396b0 0x7feb3c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb44006fd0 tx=0x7feb44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:34.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:34.792+0000 7feb4fe00700 1 -- 172.21.15.94:0/867348139 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb48013070 con 0x7feb581084e0 2024-04-25T14:52:35.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:35.023+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) -- 0x7feb38000bc0 con 0x7feb3c0396b0 2024-04-25T14:52:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:35 smithi094 ceph-mon[27862]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T14:52:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:35 smithi094 ceph-mon[27862]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:52:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:35 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:35 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:35 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:36 smithi094 ceph-mon[27862]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:36 smithi094 ceph-mon[27862]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:52:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:36 smithi094 ceph-mon[27862]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:36 smithi094 ceph-mon[27862]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:37.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:37 smithi094 ceph-mon[27862]: Deploying cephadm binary to smithi138 2024-04-25T14:52:38.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:38 smithi094 ceph-mon[27862]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:39.694 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-04-25T14:52:39.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.693+0000 7feb4fe00700 1 -- 172.21.15.94:0/867348139 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7feb38000bc0 con 0x7feb3c0396b0 2024-04-25T14:52:39.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb3c0396b0 msgr2=0x7feb3c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:39.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb3c0396b0 0x7feb3c03bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb44006fd0 tx=0x7feb44006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:39.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 msgr2=0x7feb58197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:39.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb58197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb480056d0 tx=0x7feb480057b0 comp rx=0 tx=0).stop 2024-04-25T14:52:39.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 shutdown_connections 2024-04-25T14:52:39.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7feb3c0396b0 0x7feb3c03bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:39.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 --2- 172.21.15.94:0/867348139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb581084e0 0x7feb58197eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:39.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 >> 172.21.15.94:0/867348139 conn(0x7feb580fdf80 msgr2=0x7feb5810a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:39.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 shutdown_connections 2024-04-25T14:52:39.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:39.696+0000 7feb5e200700 1 -- 172.21.15.94:0/867348139 wait complete. 2024-04-25T14:52:40.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T14:52:40.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:40 smithi094 ceph-mon[27862]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:40.840 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:52:41.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:41 smithi094 ceph-mon[27862]: Added host smithi138 2024-04-25T14:52:41.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:42.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:42 smithi094 ceph-mon[27862]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:42.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.919+0000 7f753b200700 1 -- 172.21.15.94:0/3094219189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 msgr2=0x7f753c1028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:42.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.919+0000 7f753b200700 1 --2- 172.21.15.94:0/3094219189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c1028f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f752c009910 tx=0x7f752c009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:42.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.919+0000 7f753b200700 1 -- 172.21.15.94:0/3094219189 shutdown_connections 2024-04-25T14:52:42.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.919+0000 7f753b200700 1 --2- 172.21.15.94:0/3094219189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c1028f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:42.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.919+0000 7f753b200700 1 -- 172.21.15.94:0/3094219189 >> 172.21.15.94:0/3094219189 conn(0x7f753c0fdde0 msgr2=0x7f753c100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:42.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.920+0000 7f753b200700 1 -- 172.21.15.94:0/3094219189 shutdown_connections 2024-04-25T14:52:42.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.920+0000 7f753b200700 1 -- 172.21.15.94:0/3094219189 wait complete. 2024-04-25T14:52:42.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.921+0000 7f753b200700 1 Processor -- start 2024-04-25T14:52:42.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.921+0000 7f753b200700 1 -- start start 2024-04-25T14:52:42.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.921+0000 7f753b200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:42.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.921+0000 7f753b200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10c790 con 0x7f753c102510 2024-04-25T14:52:42.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.921+0000 7f7539e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:42.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.921+0000 7f7539e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57228/0 (socket says 172.21.15.94:57228) 2024-04-25T14:52:42.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.922+0000 7f7539e00700 1 -- 172.21.15.94:0/4251123089 learned_addr learned my addr 172.21.15.94:0/4251123089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:42.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.922+0000 7f7539e00700 1 -- 172.21.15.94:0/4251123089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c076c30 con 0x7f753c102510 2024-04-25T14:52:42.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.922+0000 7f7539e00700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c074d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f752c0056d0 tx=0x7f752c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:42.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.922+0000 7f7532a00700 1 -- 172.21.15.94:0/4251123089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f752c026070 con 0x7f753c102510 2024-04-25T14:52:42.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.922+0000 7f7532a00700 1 -- 172.21.15.94:0/4251123089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f752c00bc00 con 0x7f753c102510 2024-04-25T14:52:42.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.923+0000 7f7532a00700 1 -- 172.21.15.94:0/4251123089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f752c003920 con 0x7f753c102510 2024-04-25T14:52:42.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.923+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c0752e0 con 0x7f753c102510 2024-04-25T14:52:42.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.923+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c0757a0 con 0x7f753c102510 2024-04-25T14:52:42.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.924+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c1039f0 con 0x7f753c102510 2024-04-25T14:52:42.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.924+0000 7f7532a00700 1 -- 172.21.15.94:0/4251123089 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f752c029070 con 0x7f753c102510 2024-04-25T14:52:42.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.924+0000 7f7532a00700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7520039650 0x7f752003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:42.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.924+0000 7f7532a00700 1 -- 172.21.15.94:0/4251123089 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f752c01e070 con 0x7f753c102510 2024-04-25T14:52:42.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.925+0000 7f7539400700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7520039650 0x7f752003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:42.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.925+0000 7f7539400700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7520039650 0x7f752003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7528006fd0 tx=0x7f7528006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:42.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:42.930+0000 7f7532a00700 1 -- 172.21.15.94:0/4251123089 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f752c013070 con 0x7f753c102510 2024-04-25T14:52:43.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.159+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f753c076270 con 0x7f7520039650 2024-04-25T14:52:43.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.161+0000 7f7532a00700 1 -- 172.21.15.94:0/4251123089 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f753c076270 con 0x7f7520039650 2024-04-25T14:52:43.162 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:52:43.162 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.137", "hostname": "smithi137", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.164+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7520039650 msgr2=0x7f752003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.164+0000 7f753b200700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7520039650 0x7f752003bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7528006fd0 tx=0x7f7528006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.164+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 msgr2=0x7f753c074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.164+0000 7f753b200700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c074d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f752c0056d0 tx=0x7f752c0057b0 comp rx=0 tx=0).stop 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.165+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 shutdown_connections 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.165+0000 7f753b200700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7520039650 0x7f752003bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.165+0000 7f753b200700 1 --2- 172.21.15.94:0/4251123089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f753c102510 0x7f753c074d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.165+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 >> 172.21.15.94:0/4251123089 conn(0x7f753c0fdde0 msgr2=0x7f753c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.165+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 shutdown_connections 2024-04-25T14:52:43.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:43.165+0000 7f753b200700 1 -- 172.21.15.94:0/4251123089 wait complete. 2024-04-25T14:52:43.851 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:43.911 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T14:52:43.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T14:52:44.346 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:52:44.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:44 smithi094 ceph-mon[27862]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:52:44.821 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:44 smithi094 ceph-mon[27862]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:46.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.385+0000 7f00fae00700 1 -- 172.21.15.94:0/2328524141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 msgr2=0x7f00f4075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:46.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.385+0000 7f00fae00700 1 --2- 172.21.15.94:0/2328524141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f4075250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009910 tx=0x7f00e4009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:46.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.385+0000 7f00fae00700 1 -- 172.21.15.94:0/2328524141 shutdown_connections 2024-04-25T14:52:46.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.386+0000 7f00fae00700 1 --2- 172.21.15.94:0/2328524141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f4075250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:46.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.386+0000 7f00fae00700 1 -- 172.21.15.94:0/2328524141 >> 172.21.15.94:0/2328524141 conn(0x7f00f40fdf80 msgr2=0x7f00f41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:46.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.386+0000 7f00fae00700 1 -- 172.21.15.94:0/2328524141 shutdown_connections 2024-04-25T14:52:46.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.386+0000 7f00fae00700 1 -- 172.21.15.94:0/2328524141 wait complete. 2024-04-25T14:52:46.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.387+0000 7f00fae00700 1 Processor -- start 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.387+0000 7f00fae00700 1 -- start start 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.387+0000 7f00fae00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.387+0000 7f00fae00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f00f410ebb0 con 0x7f00f4074e70 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.388+0000 7f00f3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.388+0000 7f00f3e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57248/0 (socket says 172.21.15.94:57248) 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.388+0000 7f00f3e00700 1 -- 172.21.15.94:0/1258374961 learned_addr learned my addr 172.21.15.94:0/1258374961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.388+0000 7f00f3e00700 1 -- 172.21.15.94:0/1258374961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f419a640 con 0x7f00f4074e70 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.388+0000 7f00f3e00700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f419a100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006010 tx=0x7f00e4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.389+0000 7f00f1600700 1 -- 172.21.15.94:0/1258374961 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e4026070 con 0x7f00f4074e70 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.389+0000 7f00f1600700 1 -- 172.21.15.94:0/1258374961 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00e400bc00 con 0x7f00f4074e70 2024-04-25T14:52:46.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.389+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f419a8d0 con 0x7f00f4074e70 2024-04-25T14:52:46.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.389+0000 7f00f1600700 1 -- 172.21.15.94:0/1258374961 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00e4003910 con 0x7f00f4074e70 2024-04-25T14:52:46.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.389+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f419ad30 con 0x7f00f4074e70 2024-04-25T14:52:46.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.390+0000 7f00f1600700 1 -- 172.21.15.94:0/1258374961 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f00e4029070 con 0x7f00f4074e70 2024-04-25T14:52:46.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.390+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f4076520 con 0x7f00f4074e70 2024-04-25T14:52:46.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.390+0000 7f00f1600700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00d8039700 0x7f00d803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:46.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.390+0000 7f00f1600700 1 -- 172.21.15.94:0/1258374961 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f00e401e070 con 0x7f00f4074e70 2024-04-25T14:52:46.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.391+0000 7f00eb400700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00d8039700 0x7f00d803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:46.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.391+0000 7f00eb400700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00d8039700 0x7f00d803bbc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00dc006fd0 tx=0x7f00dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:46.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.396+0000 7f00f1600700 1 -- 172.21.15.94:0/1258374961 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f00e4013070 con 0x7f00f4074e70 2024-04-25T14:52:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:46.629+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f00f419b460 con 0x7f00f4074e70 2024-04-25T14:52:46.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:46.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:52:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:52:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.477+0000 7f00f1600700 1 -- 172.21.15.94:0/1258374961 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f00e400fe20 con 0x7f00f4074e70 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00d8039700 msgr2=0x7f00d803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00d8039700 0x7f00d803bbc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f00dc006fd0 tx=0x7f00dc006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 msgr2=0x7f00f419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f419a100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00e4006010 tx=0x7f00e4005780 comp rx=0 tx=0).stop 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 shutdown_connections 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00d8039700 0x7f00d803bbc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 --2- 172.21.15.94:0/1258374961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4074e70 0x7f00f419a100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.482+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 >> 172.21.15.94:0/1258374961 conn(0x7f00f40fdf80 msgr2=0x7f00f41066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.483+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 shutdown_connections 2024-04-25T14:52:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:52:47.483+0000 7f00fae00700 1 -- 172.21.15.94:0/1258374961 wait complete. 2024-04-25T14:52:47.486 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-04-25T14:52:47.859 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T14:52:47.860 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:52:47.860 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:47.860 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:52:47.860 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1258374961' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:52:47.860 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:47.860 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:47.860 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:47 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:48.233 INFO:tasks.cephadm:Adding mon.a on smithi094 2024-04-25T14:52:48.233 INFO:tasks.cephadm:Adding mon.b on smithi137 2024-04-25T14:52:48.233 INFO:tasks.cephadm:Adding mon.c on smithi138 2024-04-25T14:52:48.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c' 2024-04-25T14:52:48.476 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:52:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:48 smithi094 ceph-mon[27862]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:48 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1258374961' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:52:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:48 smithi094 ceph-mon[27862]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:52:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 -- 172.21.15.138:0/307998019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 msgr2=0x7f39441088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:49.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 --2- 172.21.15.138:0/307998019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f39441088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3938009910 tx=0x7f3938009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 -- 172.21.15.138:0/307998019 shutdown_connections 2024-04-25T14:52:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 --2- 172.21.15.138:0/307998019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f39441088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 -- 172.21.15.138:0/307998019 >> 172.21.15.138:0/307998019 conn(0x7f39440fdf80 msgr2=0x7f39441003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 -- 172.21.15.138:0/307998019 shutdown_connections 2024-04-25T14:52:49.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 -- 172.21.15.138:0/307998019 wait complete. 2024-04-25T14:52:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 Processor -- start 2024-04-25T14:52:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.950+0000 7f394a9b2700 1 -- start start 2024-04-25T14:52:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.951+0000 7f394a9b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f3944197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.951+0000 7f394a9b2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f394410c940 con 0x7f39441084e0 2024-04-25T14:52:49.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.951+0000 7f3943fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f3944197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:49.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.951+0000 7f3943fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f3944197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.138:58038/0 (socket says 172.21.15.138:58038) 2024-04-25T14:52:49.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.951+0000 7f3943fff700 1 -- 172.21.15.138:0/823298927 learned_addr learned my addr 172.21.15.138:0/823298927 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:52:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.952+0000 7f3943fff700 1 -- 172.21.15.138:0/823298927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39441983f0 con 0x7f39441084e0 2024-04-25T14:52:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.952+0000 7f3943fff700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f3944197eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39380056d0 tx=0x7f39380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.952+0000 7f39417fa700 1 -- 172.21.15.138:0/823298927 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3938026070 con 0x7f39441084e0 2024-04-25T14:52:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.953+0000 7f39417fa700 1 -- 172.21.15.138:0/823298927 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f393800bba0 con 0x7f39441084e0 2024-04-25T14:52:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.953+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39441986e0 con 0x7f39441084e0 2024-04-25T14:52:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.953+0000 7f39417fa700 1 -- 172.21.15.138:0/823298927 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3938003920 con 0x7f39441084e0 2024-04-25T14:52:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.953+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394419c780 con 0x7f39441084e0 2024-04-25T14:52:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.954+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3944103b90 con 0x7f39441084e0 2024-04-25T14:52:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.954+0000 7f39417fa700 1 -- 172.21.15.138:0/823298927 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3938029070 con 0x7f39441084e0 2024-04-25T14:52:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.954+0000 7f39417fa700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f392c039650 0x7f392c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:49.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.954+0000 7f39417fa700 1 -- 172.21.15.138:0/823298927 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f393801e070 con 0x7f39441084e0 2024-04-25T14:52:49.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.955+0000 7f39437fe700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f392c039650 0x7f392c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:49.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.956+0000 7f39437fe700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f392c039650 0x7f392c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3934006fd0 tx=0x7f3934006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:49.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:49.971+0000 7f39417fa700 1 -- 172.21.15.138:0/823298927 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3938030020 con 0x7f39441084e0 2024-04-25T14:52:50.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.178+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}) -- 0x7f3944199210 con 0x7f392c039650 2024-04-25T14:52:50.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.192+0000 7f39417fa700 1 -- 172.21.15.138:0/823298927 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3944199210 con 0x7f392c039650 2024-04-25T14:52:50.193 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mon update... 2024-04-25T14:52:50.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f392c039650 msgr2=0x7f392c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:50.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f392c039650 0x7f392c03bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3934006fd0 tx=0x7f3934006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:50.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 msgr2=0x7f3944197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:50.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f3944197eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f39380056d0 tx=0x7f39380057b0 comp rx=0 tx=0).stop 2024-04-25T14:52:50.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 shutdown_connections 2024-04-25T14:52:50.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f392c039650 0x7f392c03bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:50.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 --2- 172.21.15.138:0/823298927 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39441084e0 0x7f3944197eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:50.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 >> 172.21.15.138:0/823298927 conn(0x7f39440fdf80 msgr2=0x7f394410a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:50.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 shutdown_connections 2024-04-25T14:52:50.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:50.196+0000 7f394a9b2700 1 -- 172.21.15.138:0/823298927 wait complete. 2024-04-25T14:52:50.850 DEBUG:teuthology.orchestra.run.smithi137:mon.b> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T14:52:50.853 DEBUG:teuthology.orchestra.run.smithi138:mon.c> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T14:52:50.855 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:52:50.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:52:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:51.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:51 smithi094 ceph-mon[27862]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:51.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:51 smithi094 ceph-mon[27862]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c;count:3 2024-04-25T14:52:51.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:51 smithi094 ceph-mon[27862]: Deploying daemon mon.c on smithi138 2024-04-25T14:52:52.088 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:52:52.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:52 smithi094 ceph-mon[27862]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:54.214 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:54 smithi138 systemd[1]: Starting Ceph mon.c for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:52:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 -- 172.21.15.138:0/929400534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c1084e0 msgr2=0x7fb07c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 --2- 172.21.15.138:0/929400534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c1084e0 0x7fb07c1088c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb074009910 tx=0x7fb074009c20 comp rx=0 tx=0).stop 2024-04-25T14:52:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 -- 172.21.15.138:0/929400534 shutdown_connections 2024-04-25T14:52:54.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 --2- 172.21.15.138:0/929400534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c1084e0 0x7fb07c1088c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 -- 172.21.15.138:0/929400534 >> 172.21.15.138:0/929400534 conn(0x7fb07c0fdf80 msgr2=0x7fb07c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 -- 172.21.15.138:0/929400534 shutdown_connections 2024-04-25T14:52:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 -- 172.21.15.138:0/929400534 wait complete. 2024-04-25T14:52:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 Processor -- start 2024-04-25T14:52:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.927+0000 7fb0833ea700 1 -- start start 2024-04-25T14:52:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.928+0000 7fb0833ea700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c074d40 0x7fb07c075120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:54.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.928+0000 7fb0833ea700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c10c940 con 0x7fb07c074d40 2024-04-25T14:52:54.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.928+0000 7fb081186700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c074d40 0x7fb07c075120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:54.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.928+0000 7fb081186700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c074d40 0x7fb07c075120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.138:58048/0 (socket says 172.21.15.138:58048) 2024-04-25T14:52:54.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.928+0000 7fb081186700 1 -- 172.21.15.138:0/1352157478 learned_addr learned my addr 172.21.15.138:0/1352157478 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:52:54.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.929+0000 7fb081186700 1 -- 172.21.15.138:0/1352157478 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c076be0 con 0x7fb07c074d40 2024-04-25T14:52:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.929+0000 7fb081186700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c074d40 0x7fb07c075120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b5c0 tx=0x7fb074005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.929+0000 7fb07a7fc700 1 -- 172.21.15.138:0/1352157478 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074026070 con 0x7fb07c074d40 2024-04-25T14:52:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.929+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c075660 con 0x7fb07c074d40 2024-04-25T14:52:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.929+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c075b20 con 0x7fb07c074d40 2024-04-25T14:52:54.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.929+0000 7fb07a7fc700 1 -- 172.21.15.138:0/1352157478 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07400b6f0 con 0x7fb07c074d40 2024-04-25T14:52:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.930+0000 7fb07a7fc700 1 -- 172.21.15.138:0/1352157478 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074030060 con 0x7fb07c074d40 2024-04-25T14:52:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.930+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07c103b90 con 0x7fb07c074d40 2024-04-25T14:52:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.931+0000 7fb07a7fc700 1 -- 172.21.15.138:0/1352157478 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fb074029070 con 0x7fb07c074d40 2024-04-25T14:52:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.931+0000 7fb07a7fc700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb064039650 0x7fb06403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:52:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.931+0000 7fb07a7fc700 1 -- 172.21.15.138:0/1352157478 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb07401e070 con 0x7fb07c074d40 2024-04-25T14:52:54.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.932+0000 7fb080985700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb064039650 0x7fb06403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:52:54.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.933+0000 7fb080985700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb064039650 0x7fb06403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb070006fd0 tx=0x7fb070006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:52:54.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:54.948+0000 7fb07a7fc700 1 -- 172.21.15.138:0/1352157478 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb074013070 con 0x7fb07c074d40 2024-04-25T14:52:54.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:52:54 smithi094 ceph-mon[27862]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.238+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb07c0640d0 con 0x7fb07c074d40 2024-04-25T14:52:55.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.239+0000 7fb07a7fc700 1 -- 172.21.15.138:0/1352157478 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb07c0640d0 con 0x7fb07c074d40 2024-04-25T14:52:55.241 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:52:55.241 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","modified":"2024-04-25T14:50:41.313017Z","created":"2024-04-25T14:50:41.313017Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T14:52:55.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb064039650 msgr2=0x7fb06403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:55.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb064039650 0x7fb06403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb070006fd0 tx=0x7fb070006e40 comp rx=0 tx=0).stop 2024-04-25T14:52:55.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c074d40 msgr2=0x7fb07c075120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:52:55.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c074d40 0x7fb07c075120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b5c0 tx=0x7fb074005780 comp rx=0 tx=0).stop 2024-04-25T14:52:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 shutdown_connections 2024-04-25T14:52:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb064039650 0x7fb06403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 --2- 172.21.15.138:0/1352157478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb07c074d40 0x7fb07c075120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:52:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 >> 172.21.15.138:0/1352157478 conn(0x7fb07c0fdf80 msgr2=0x7fb07c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:52:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 shutdown_connections 2024-04-25T14:52:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:52:55.241+0000 7fb0833ea700 1 -- 172.21.15.138:0/1352157478 wait complete. 2024-04-25T14:52:55.243 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T14:52:55.384 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 podman[32842]: 2024-04-25 14:52:55.035155091 +0000 UTC m=+0.022476356 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:52:55.385 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 podman[32842]: 2024-04-25 14:52:55.135874091 +0000 UTC m=+0.123195330 container create ef5fb5e28152dc3fb37c5d781771bcc5234d45846c1bcdcf0c3be3629882eaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:52:55.385 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 podman[32842]: 2024-04-25 14:52:55.30833866 +0000 UTC m=+0.295659899 container init ef5fb5e28152dc3fb37c5d781771bcc5234d45846c1bcdcf0c3be3629882eaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T14:52:55.385 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 podman[32842]: 2024-04-25 14:52:55.311808618 +0000 UTC m=+0.299129869 container start ef5fb5e28152dc3fb37c5d781771bcc5234d45846c1bcdcf0c3be3629882eaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T14:52:55.385 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:52:55.385 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T14:52:55.386 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pidfile_write: ignore empty --pid-file 2024-04-25T14:52:55.386 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: load: jerasure load: lrc 2024-04-25T14:52:55.386 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:52:55.386 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Git sha 0 2024-04-25T14:52:55.386 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T14:52:55.386 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: DB SUMMARY 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: DB Session ID: FWEKBXHEJ9KVCPSXC4LH 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:52:55.387 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.env: 0x56213364bcc0 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.info_log: 0x562135e67be0 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.statistics: (nil) 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.use_fsync: 0 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:52:55.388 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.db_log_dir: 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.wal_dir: 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:52:55.389 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.write_buffer_manager: 0x562135dfb0e0 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:52:55.390 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.unordered_write: 0 2024-04-25T14:52:55.391 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.row_cache: None 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.wal_filter: None 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.wal_compression: 0 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:52:55.392 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:52:55.393 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:52:55.393 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:52:55.393 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:52:55.393 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:52:55.393 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:52:55.393 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:52:55.393 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:52:55.394 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_open_files: -1 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Compression algorithms supported: 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kZSTD supported: 0 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:52:55.395 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.merge_operator: 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_filter: None 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562135e67dc0) 2024-04-25T14:52:55.396 INFO:journalctl@ceph.mon.c.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:52:55.397 INFO:journalctl@ceph.mon.c.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:52:55.397 INFO:journalctl@ceph.mon.c.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:52:55.397 INFO:journalctl@ceph.mon.c.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:52:55.397 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_type: 0 2024-04-25T14:52:55.397 INFO:journalctl@ceph.mon.c.smithi138.stdout: data_block_index_type: 0 2024-04-25T14:52:55.397 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_shortening: 1 2024-04-25T14:52:55.397 INFO:journalctl@ceph.mon.c.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: checksum: 4 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: no_block_cache: 0 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache: 0x562135e2d1f0 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_options: 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: capacity : 536870912 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: num_shard_bits : 4 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: strict_capacity_limit : 0 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_cache_compressed: (nil) 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: persistent_cache: (nil) 2024-04-25T14:52:55.398 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_size: 4096 2024-04-25T14:52:55.399 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_size_deviation: 10 2024-04-25T14:52:55.400 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_restart_interval: 16 2024-04-25T14:52:55.400 INFO:journalctl@ceph.mon.c.smithi138.stdout: index_block_restart_interval: 1 2024-04-25T14:52:55.400 INFO:journalctl@ceph.mon.c.smithi138.stdout: metadata_block_size: 4096 2024-04-25T14:52:55.400 INFO:journalctl@ceph.mon.c.smithi138.stdout: partition_filters: 0 2024-04-25T14:52:55.401 INFO:journalctl@ceph.mon.c.smithi138.stdout: use_delta_encoding: 1 2024-04-25T14:52:55.401 INFO:journalctl@ceph.mon.c.smithi138.stdout: filter_policy: bloomfilter 2024-04-25T14:52:55.401 INFO:journalctl@ceph.mon.c.smithi138.stdout: whole_key_filtering: 1 2024-04-25T14:52:55.401 INFO:journalctl@ceph.mon.c.smithi138.stdout: verify_compression: 0 2024-04-25T14:52:55.401 INFO:journalctl@ceph.mon.c.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:52:55.401 INFO:journalctl@ceph.mon.c.smithi138.stdout: format_version: 5 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout: enable_index_compression: 1 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout: block_align: 0 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout: max_auto_readahead_size: 262144 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout: prepopulate_block_cache: 0 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression: NoCompression 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:52:55.402 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.num_levels: 7 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:52:55.403 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:52:55.404 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:52:55.405 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:52:55.405 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:52:55.405 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:52:55.405 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:52:55.405 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:52:55.405 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:52:55.405 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:52:55.406 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:52:55.407 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:52:55.408 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.ttl: 2592000 2024-04-25T14:52:55.409 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:52:55.409 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:52:55.409 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:52:55.409 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:52:55.409 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:52:55.409 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:52:55.409 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2497f961-47e3-4a47-ac0d-7f8f85bf3579 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056775341591, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T14:52:55.410 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056775342013, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714056775, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2497f961-47e3-4a47-ac0d-7f8f85bf3579", "db_session_id": "FWEKBXHEJ9KVCPSXC4LH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056775342094, "job": 1, "event": "recovery_finished"} 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562135f42000 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: DB pointer 0x562135f2c000 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:52:55.411 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** DB Stats ** 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:52:55.412 INFO:journalctl@ceph.mon.c.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:52:55.413 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: Block cache BinnedLRUCache@0x562135e2d1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T14:52:55.414 INFO:journalctl@ceph.mon.c.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: using public_addr v2:172.21.15.138:0/0 -> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: starting mon.c rank -1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(???) e0 preinit fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout: e1 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout: legacy client fscid: -1 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout: 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout: No filesystems configured 2024-04-25T14:52:55.415 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:52:55.416 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Added host smithi137 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.417 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:52:55.418 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Deploying cephadm binary to smithi138 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.419 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Added host smithi138 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T14:52:55.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1258374961' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1258374961' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:52:55.421 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c;count:3 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: Deploying daemon mon.c on smithi138 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 ceph-mon[32875]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 bash[32842]: ef5fb5e28152dc3fb37c5d781771bcc5234d45846c1bcdcf0c3be3629882eaa6 2024-04-25T14:52:55.422 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:52:55 smithi138 systemd[1]: Started Ceph mon.c for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:52:56.932 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:52:56.933 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:52:57.184 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:53:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: mon.a calling monitor election 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: mon.c calling monitor election 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:53:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: monmap epoch 2 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: last_changed 2024-04-25T14:52:55.375334+0000 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: min_mon_release 19 (squid) 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: election_strategy: 1 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: fsmap 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: mgrmap e13: x(active, since 59s) 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: overall HEALTH_OK 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:00.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:00.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: mon.a calling monitor election 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: mon.c calling monitor election 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:53:00.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: monmap epoch 2 2024-04-25T14:53:00.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:00.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: last_changed 2024-04-25T14:52:55.375334+0000 2024-04-25T14:53:00.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:00.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: min_mon_release 19 (squid) 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: election_strategy: 1 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: fsmap 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: mgrmap e13: x(active, since 59s) 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: overall HEALTH_OK 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:00.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:01.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.627+0000 7f52b1f1e700 1 -- 172.21.15.138:0/3921740375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 msgr2=0x7f5294005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:01.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.627+0000 7f52b1f1e700 1 --2- 172.21.15.138:0/3921740375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 0x7f5294005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52ac06ad30 tx=0x7f529c00feb0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.627+0000 7f52b1f1e700 1 -- 172.21.15.138:0/3921740375 shutdown_connections 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.628+0000 7f52b1f1e700 1 --2- 172.21.15.138:0/3921740375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 0x7f5294005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.628+0000 7f52b1f1e700 1 -- 172.21.15.138:0/3921740375 >> 172.21.15.138:0/3921740375 conn(0x7f52ac0fcf80 msgr2=0x7f52ac0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.628+0000 7f52b1f1e700 1 -- 172.21.15.138:0/3921740375 shutdown_connections 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.628+0000 7f52b1f1e700 1 -- 172.21.15.138:0/3921740375 wait complete. 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.628+0000 7f52b1f1e700 1 Processor -- start 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.628+0000 7f52b1f1e700 1 -- start start 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52b1f1e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 0x7f52ac198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52b1f1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 0x7f52ac19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52b1f1e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac10a4a0 con 0x7f52940035c0 2024-04-25T14:53:01.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52b1f1e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac10a320 con 0x7f52ac198550 2024-04-25T14:53:01.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52aaffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 0x7f52ac19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52aaffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 0x7f52ac19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45496/0 (socket says 172.21.15.138:45496) 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52aaffd700 1 -- 172.21.15.138:0/655939434 learned_addr learned my addr 172.21.15.138:0/655939434 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52aaffd700 1 -- 172.21.15.138:0/655939434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 msgr2=0x7f52ac19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52aaffd700 1 -- 172.21.15.138:0/655939434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 msgr2=0x7f52ac19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52aaffd700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 0x7f52ac19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52ab7fe700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 0x7f52ac198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.629+0000 7f52aaffd700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 0x7f52ac19c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.630+0000 7f52ab7fe700 1 -- 172.21.15.138:0/655939434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 msgr2=0x7f52ac19c8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.630+0000 7f52ab7fe700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 0x7f52ac19c8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.630+0000 7f52ab7fe700 1 -- 172.21.15.138:0/655939434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac198b90 con 0x7f52940035c0 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.630+0000 7f52ab7fe700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 0x7f52ac198010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f529c010010 tx=0x7f529c0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:01.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.630+0000 7f52a8ff9700 1 -- 172.21.15.138:0/655939434 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f529c013b00 con 0x7f52940035c0 2024-04-25T14:53:01.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.630+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac19ce20 con 0x7f52940035c0 2024-04-25T14:53:01.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.630+0000 7f52a8ff9700 1 -- 172.21.15.138:0/655939434 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0044e0 con 0x7f52940035c0 2024-04-25T14:53:01.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.631+0000 7f52a8ff9700 1 -- 172.21.15.138:0/655939434 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f529c003de0 con 0x7f52940035c0 2024-04-25T14:53:01.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.631+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac19d380 con 0x7f52940035c0 2024-04-25T14:53:01.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.631+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52ac19d6f0 con 0x7f52940035c0 2024-04-25T14:53:01.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.632+0000 7f52a8ff9700 1 -- 172.21.15.138:0/655939434 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f529c023070 con 0x7f52940035c0 2024-04-25T14:53:01.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.632+0000 7f52a8ff9700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52980393f0 0x7f529803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:01.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.632+0000 7f52a8ff9700 1 -- 172.21.15.138:0/655939434 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f529c020050 con 0x7f52940035c0 2024-04-25T14:53:01.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.633+0000 7f52aaffd700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52980393f0 0x7f529803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:01.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.633+0000 7f52aaffd700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52980393f0 0x7f529803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009180 tx=0x7f52a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:01.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.647+0000 7f52a8ff9700 1 -- 172.21.15.138:0/655939434 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f529c028020 con 0x7f52940035c0 2024-04-25T14:53:01.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.925+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f52ac199370 con 0x7f52940035c0 2024-04-25T14:53:01.926 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:01.927 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:01.927 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:01.927 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:01 smithi138 ceph-mon[32875]: Deploying daemon mon.b on smithi137 2024-04-25T14:53:01.927 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:01.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.926+0000 7f52a8ff9700 1 -- 172.21.15.138:0/655939434 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f529c00f890 con 0x7f52940035c0 2024-04-25T14:53:01.927 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:53:01.927 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":2,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","modified":"2024-04-25T14:52:55.375334Z","created":"2024-04-25T14:50:41.313017Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:53:01.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52980393f0 msgr2=0x7f529803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52980393f0 0x7f529803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009180 tx=0x7f52a0008040 comp rx=0 tx=0).stop 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 msgr2=0x7f52ac198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 0x7f52ac198010 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f529c010010 tx=0x7f529c0046f0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 shutdown_connections 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52980393f0 0x7f529803b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52940035c0 0x7f52ac198010 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 --2- 172.21.15.138:0/655939434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52ac198550 0x7f52ac19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 >> 172.21.15.138:0/655939434 conn(0x7f52ac0fcf80 msgr2=0x7f52ac0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 shutdown_connections 2024-04-25T14:53:01.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:01.929+0000 7f52b1f1e700 1 -- 172.21.15.138:0/655939434 wait complete. 2024-04-25T14:53:01.931 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 2 2024-04-25T14:53:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:01 smithi094 ceph-mon[27862]: Deploying daemon mon.b on smithi137 2024-04-25T14:53:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:02.483 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:02 smithi138 ceph-mon[32875]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:02.484 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:02 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/655939434' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:53:02.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:02 smithi094 ceph-mon[27862]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:02 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/655939434' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:53:03.509 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T14:53:03.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T14:53:03.751 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:53:04.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 podman[32420]: 2024-04-25 14:53:04.166517376 +0000 UTC m=+0.127155197 container create af3d393348f445563be78afcc8bdc4348aa2dd6980c0927e3fdadf29488270b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T14:53:04.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 podman[32420]: 2024-04-25 14:53:04.369133 +0000 UTC m=+0.329770821 container init af3d393348f445563be78afcc8bdc4348aa2dd6980c0927e3fdadf29488270b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 podman[32420]: 2024-04-25 14:53:04.372190468 +0000 UTC m=+0.332828300 container start af3d393348f445563be78afcc8bdc4348aa2dd6980c0927e3fdadf29488270b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pidfile_write: ignore empty --pid-file 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: load: jerasure load: lrc 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: RocksDB version: 7.9.2 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Git sha 0 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: DB SUMMARY 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: DB Session ID: RZ2L8373I8MDD4WPQV6I 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: CURRENT file: CURRENT 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T14:53:04.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.error_if_exists: 0 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.create_if_missing: 0 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.paranoid_checks: 1 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.env: 0x5650f2341cc0 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.info_log: 0x5650f3c67be0 2024-04-25T14:53:04.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.statistics: (nil) 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.use_fsync: 0 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_log_file_size: 0 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.allow_fallocate: 1 2024-04-25T14:53:04.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.use_direct_reads: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.db_log_dir: 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.wal_dir: 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T14:53:04.737 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.write_buffer_manager: 0x5650f3bfb0e0 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.unordered_write: 0 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T14:53:04.738 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.row_cache: None 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.wal_filter: None 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.two_write_queues: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.wal_compression: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.atomic_flush: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.log_readahead_size: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T14:53:04.739 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_background_jobs: 2 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_background_compactions: -1 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_subcompactions: 1 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T14:53:04.740 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_open_files: -1 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_background_flushes: -1 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Compression algorithms supported: 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kZSTD supported: 0 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kXpressCompression supported: 0 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kZlibCompression supported: 1 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kSnappyCompression supported: 1 2024-04-25T14:53:04.741 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kLZ4Compression supported: 1 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: kBZip2Compression supported: 0 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.merge_operator: 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_filter: None 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T14:53:04.742 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5650f3c67dc0) 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: cache_index_and_filter_blocks: 1 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: pin_top_level_index_and_filter: 1 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_type: 0 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: data_block_index_type: 0 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_shortening: 1 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: checksum: 4 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: no_block_cache: 0 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache: 0x5650f3c2d1f0 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_name: BinnedLRUCache 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_options: 2024-04-25T14:53:04.743 INFO:journalctl@ceph.mon.b.smithi137.stdout: capacity : 536870912 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: num_shard_bits : 4 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: strict_capacity_limit : 0 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: high_pri_pool_ratio: 0.000 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_cache_compressed: (nil) 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: persistent_cache: (nil) 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_size: 4096 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_size_deviation: 10 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_restart_interval: 16 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: index_block_restart_interval: 1 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: metadata_block_size: 4096 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: partition_filters: 0 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: use_delta_encoding: 1 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: filter_policy: bloomfilter 2024-04-25T14:53:04.744 INFO:journalctl@ceph.mon.b.smithi137.stdout: whole_key_filtering: 1 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: verify_compression: 0 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: read_amp_bytes_per_bit: 0 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: format_version: 5 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: enable_index_compression: 1 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: block_align: 0 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: max_auto_readahead_size: 262144 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: prepopulate_block_cache: 0 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: initial_auto_readahead_size: 8192 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression: NoCompression 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T14:53:04.745 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.num_levels: 7 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T14:53:04.746 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T14:53:04.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T14:53:04.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T14:53:04.749 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.inplace_update_support: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.bloom_locality: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.max_successive_merges: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.ttl: 2592000 2024-04-25T14:53:04.750 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.enable_blob_files: false 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.min_blob_size: 0 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T14:53:04.751 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1807d019-c974-4b22-9ca8-9b407f79fad7 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056784401436, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056784401859, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714056784, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1807d019-c974-4b22-9ca8-9b407f79fad7", "db_session_id": "RZ2L8373I8MDD4WPQV6I", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714056784401939, "job": 1, "event": "recovery_finished"} 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5650f3d42000 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: DB pointer 0x5650f3d2c000 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T14:53:04.752 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** DB Stats ** 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** Compaction Stats [default] ** 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:53:04.753 INFO:journalctl@ceph.mon.b.smithi137.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** Compaction Stats [default] ** 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T14:53:04.754 INFO:journalctl@ceph.mon.b.smithi137.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: Cumulative compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: Interval compaction: 0.00 GB write, 0.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: Block cache BinnedLRUCache@0x5650f3c2d1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: using public_addr v2:172.21.15.137:0/0 -> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] 2024-04-25T14:53:04.755 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: starting mon.b rank -1 at public addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] at bind addrs [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(???) e0 preinit fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout: e1 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout: legacy client fscid: -1 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout: 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout: No filesystems configured 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T14:53:04.756 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Added host smithi137 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi137", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:53:04.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Deploying cephadm binary to smithi138 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Added host smithi138 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:53:04.759 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1258374961' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T14:53:04.760 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1258374961' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:04.761 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:04.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c;count:3 2024-04-25T14:53:04.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Deploying daemon mon.c on smithi138 2024-04-25T14:53:04.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:04.762 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.a calling monitor election 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.c calling monitor election 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: monmap epoch 2 2024-04-25T14:53:04.763 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: last_changed 2024-04-25T14:52:55.375334+0000 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: min_mon_release 19 (squid) 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: election_strategy: 1 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: fsmap 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mgrmap e13: x(active, since 59s) 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: overall HEALTH_OK 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:04.764 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: Deploying daemon mon.b on smithi137 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/655939434' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 ceph-mon[32439]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 bash[32420]: af3d393348f445563be78afcc8bdc4348aa2dd6980c0927e3fdadf29488270b3 2024-04-25T14:53:04.765 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:04 smithi137 systemd[1]: Started Ceph mon.b for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.623+0000 7f618acd0700 1 -- 172.21.15.138:0/237853718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 msgr2=0x7f6174005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.623+0000 7f618acd0700 1 --2- 172.21.15.138:0/237853718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 0x7f6174005a50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6184102490 tx=0x7f617800fe60 comp rx=0 tx=0).stop 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.623+0000 7f618acd0700 1 -- 172.21.15.138:0/237853718 shutdown_connections 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.623+0000 7f618acd0700 1 --2- 172.21.15.138:0/237853718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 0x7f6174005a50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.623+0000 7f618acd0700 1 -- 172.21.15.138:0/237853718 >> 172.21.15.138:0/237853718 conn(0x7f61840fe050 msgr2=0x7f6184100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.623+0000 7f618acd0700 1 -- 172.21.15.138:0/237853718 shutdown_connections 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.623+0000 7f618acd0700 1 -- 172.21.15.138:0/237853718 wait complete. 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.624+0000 7f618acd0700 1 Processor -- start 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.624+0000 7f618acd0700 1 -- start start 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.624+0000 7f618acd0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 0x7f618419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:09.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.624+0000 7f618acd0700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 0x7f618419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.624+0000 7f618acd0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 0x7f618419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.624+0000 7f618acd0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f618410a460 con 0x7f61740035c0 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618acd0700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f618410a2e0 con 0x7f618419a8c0 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618acd0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f618410a5e0 con 0x7f618419f190 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618926d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 0x7f618419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618926d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 0x7f618419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35144/0 (socket says 172.21.15.138:35144) 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618926d700 1 -- 172.21.15.138:0/1387708373 learned_addr learned my addr 172.21.15.138:0/1387708373 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6183fff700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 0x7f618419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618926d700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 msgr2=0x7f618419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618926d700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 msgr2=0x7f618419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618926d700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 0x7f618419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6188a6c700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 0x7f618419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f618926d700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 0x7f618419f640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:53:09.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6183fff700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 msgr2=0x7f618419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6183fff700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 msgr2=0x7f618419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6183fff700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 0x7f618419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6183fff700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 0x7f618419ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6188a6c700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 msgr2=0x7f618419f640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6188a6c700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 0x7f618419f640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6188a6c700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 msgr2=0x7f618419ec50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6188a6c700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 0x7f618419ec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.625+0000 7f6188a6c700 1 -- 172.21.15.138:0/1387708373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61841a6f00 con 0x7f61740035c0 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.626+0000 7f6188a6c700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 0x7f618419a380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6184102490 tx=0x7f6178003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.626+0000 7f6181ffb700 1 -- 172.21.15.138:0/1387708373 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617802b020 con 0x7f61740035c0 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.626+0000 7f6181ffb700 1 -- 172.21.15.138:0/1387708373 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f617801d440 con 0x7f61740035c0 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.626+0000 7f6181ffb700 1 -- 172.21.15.138:0/1387708373 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178022040 con 0x7f61740035c0 2024-04-25T14:53:09.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.626+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61841a70d0 con 0x7f61740035c0 2024-04-25T14:53:09.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.626+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841a7660 con 0x7f61740035c0 2024-04-25T14:53:09.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.627+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61840640a0 con 0x7f61740035c0 2024-04-25T14:53:09.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.628+0000 7f6181ffb700 1 -- 172.21.15.138:0/1387708373 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f617802e020 con 0x7f61740035c0 2024-04-25T14:53:09.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.628+0000 7f6181ffb700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6168039ae0 0x7f616803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:09.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.628+0000 7f6181ffb700 1 -- 172.21.15.138:0/1387708373 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f617804f080 con 0x7f61740035c0 2024-04-25T14:53:09.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.628+0000 7f6183fff700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6168039ae0 0x7f616803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:09.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.629+0000 7f6183fff700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6168039ae0 0x7f616803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6170009850 tx=0x7f6170008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:09.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.635+0000 7f6181ffb700 1 -- 172.21.15.138:0/1387708373 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6178035020 con 0x7f61740035c0 2024-04-25T14:53:09.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.926+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f618402df80 con 0x7f61740035c0 2024-04-25T14:53:09.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.927+0000 7f6181ffb700 1 -- 172.21.15.138:0/1387708373 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f617801c600 con 0x7f61740035c0 2024-04-25T14:53:09.928 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T14:53:09.928 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":3,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","modified":"2024-04-25T14:53:04.427253Z","created":"2024-04-25T14:50:41.313017Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:3300","nonce":0},{"type":"v1","addr":"172.21.15.137:6789","nonce":0}]},"addr":"172.21.15.137:6789/0","public_addr":"172.21.15.137:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: mon.c calling monitor election 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: mon.a calling monitor election 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.928 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: monmap epoch 3 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: last_changed 2024-04-25T14:53:04.427253+0000 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: min_mon_release 19 (squid) 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: election_strategy: 1 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T14:53:09.929 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: fsmap 2024-04-25T14:53:09.930 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:09.930 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: mgrmap e13: x(active, since 68s) 2024-04-25T14:53:09.930 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: overall HEALTH_OK 2024-04-25T14:53:09.930 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:09.930 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:09.930 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:09.930 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:09.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.929+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6168039ae0 msgr2=0x7f616803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.929+0000 7f618acd0700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6168039ae0 0x7f616803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6170009850 tx=0x7f6170008040 comp rx=0 tx=0).stop 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.929+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 msgr2=0x7f618419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.929+0000 7f618acd0700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 0x7f618419a380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6184102490 tx=0x7f6178003cd0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 shutdown_connections 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6168039ae0 0x7f616803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f618419f190 0x7f618419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61740035c0 0x7f618419a380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 --2- 172.21.15.138:0/1387708373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f618419a8c0 0x7f618419ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 >> 172.21.15.138:0/1387708373 conn(0x7f61840fe050 msgr2=0x7f61840ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 shutdown_connections 2024-04-25T14:53:09.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:09.930+0000 7f618acd0700 1 -- 172.21.15.138:0/1387708373 wait complete. 2024-04-25T14:53:09.932 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 3 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: mon.c calling monitor election 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: mon.a calling monitor election 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: monmap epoch 3 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: last_changed 2024-04-25T14:53:04.427253+0000 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: min_mon_release 19 (squid) 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: election_strategy: 1 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: fsmap 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: mgrmap e13: x(active, since 68s) 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: overall HEALTH_OK 2024-04-25T14:53:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:09.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:09.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:09.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:10.588 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:10 smithi138 ceph-mon[32875]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:10.588 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:10 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:10.588 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:10 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/1387708373' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:53:10.588 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:10 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:10.614 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T14:53:10.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T14:53:10.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:10 smithi094 ceph-mon[27862]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:10.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:10 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:10.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:10 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/1387708373' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:53:10.638 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:10 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:11.029 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:53:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: mon.c calling monitor election 2024-04-25T14:53:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: mon.a calling monitor election 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: monmap epoch 3 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:11.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: last_changed 2024-04-25T14:53:04.427253+0000 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: min_mon_release 19 (squid) 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: election_strategy: 1 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: fsmap 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: mgrmap e13: x(active, since 68s) 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: overall HEALTH_OK 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:11.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:11.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:11.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:11.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/1387708373' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T14:53:11.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:11 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:12.523 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: mon.b calling monitor election 2024-04-25T14:53:12.523 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:12.523 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T14:53:12.523 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: mon.b calling monitor election 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: mon.c calling monitor election 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: mon.a calling monitor election 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: monmap epoch 3 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: last_changed 2024-04-25T14:53:04.427253+0000 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: min_mon_release 19 (squid) 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: election_strategy: 1 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:12.524 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: fsmap 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: mgrmap e13: x(active, since 70s) 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: overall HEALTH_OK 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.525 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: mon.b calling monitor election 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: mon.b calling monitor election 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: mon.c calling monitor election 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: mon.a calling monitor election 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: monmap epoch 3 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: last_changed 2024-04-25T14:53:04.427253+0000 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:12.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: min_mon_release 19 (squid) 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: election_strategy: 1 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: fsmap 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: mgrmap e13: x(active, since 70s) 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: overall HEALTH_OK 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: mon.b calling monitor election 2024-04-25T14:53:12.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:12.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T14:53:12.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T14:53:12.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T14:53:12.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: mon.b calling monitor election 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: mon.c calling monitor election 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: mon.a calling monitor election 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: monmap epoch 3 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: last_changed 2024-04-25T14:53:04.427253+0000 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: created 2024-04-25T14:50:41.313017+0000 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: min_mon_release 19 (squid) 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: election_strategy: 1 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: 0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.c 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: 2: [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] mon.b 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: fsmap 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T14:53:12.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: mgrmap e13: x(active, since 70s) 2024-04-25T14:53:12.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: overall HEALTH_OK 2024-04-25T14:53:12.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:12.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.041+0000 7f0eb0200700 1 -- 172.21.15.94:0/3645289515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea8074de0 msgr2=0x7f0ea80751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:13.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.041+0000 7f0eb0200700 1 --2- 172.21.15.94:0/3645289515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea8074de0 0x7f0ea80751c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009910 tx=0x7f0e9c009c20 comp rx=0 tx=0).stop 2024-04-25T14:53:13.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.041+0000 7f0eb0200700 1 -- 172.21.15.94:0/3645289515 shutdown_connections 2024-04-25T14:53:13.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.041+0000 7f0eb0200700 1 --2- 172.21.15.94:0/3645289515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea8074de0 0x7f0ea80751c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.041+0000 7f0eb0200700 1 -- 172.21.15.94:0/3645289515 >> 172.21.15.94:0/3645289515 conn(0x7f0ea80fdf80 msgr2=0x7f0ea81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:13.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.041+0000 7f0eb0200700 1 -- 172.21.15.94:0/3645289515 shutdown_connections 2024-04-25T14:53:13.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.041+0000 7f0eb0200700 1 -- 172.21.15.94:0/3645289515 wait complete. 2024-04-25T14:53:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.042+0000 7f0eb0200700 1 Processor -- start 2024-04-25T14:53:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.042+0000 7f0eb0200700 1 -- start start 2024-04-25T14:53:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eb0200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea8074de0 0x7f0ea81a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eb0200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea81a4960 0x7f0ea819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eb0200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea819eb20 0x7f0ea819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:13.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eb0200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8112fa0 con 0x7f0ea819eb20 2024-04-25T14:53:13.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eb0200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8112e20 con 0x7f0ea81a4960 2024-04-25T14:53:13.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eb0200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8113120 con 0x7f0ea8074de0 2024-04-25T14:53:13.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eace00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea81a4960 0x7f0ea819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:13.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eae200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea819eb20 0x7f0ea819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:13.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eace00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea81a4960 0x7f0ea819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:51372/0 (socket says 172.21.15.94:51372) 2024-04-25T14:53:13.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0eace00700 1 -- 172.21.15.94:0/973519027 learned_addr learned my addr 172.21.15.94:0/973519027 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:53:13.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.043+0000 7f0ead800700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea8074de0 0x7f0ea81a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eace00700 1 -- 172.21.15.94:0/973519027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea8074de0 msgr2=0x7f0ea81a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eace00700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea8074de0 0x7f0ea81a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eace00700 1 -- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea819eb20 msgr2=0x7f0ea819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eace00700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea819eb20 0x7f0ea819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eace00700 1 -- 172.21.15.94:0/973519027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea81ab2f0 con 0x7f0ea81a4960 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0ead800700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea8074de0 0x7f0ea81a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eace00700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea81a4960 0x7f0ea819e5e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400b780 tx=0x7f0e9400ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0ea6000700 1 -- 172.21.15.94:0/973519027 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94004230 con 0x7f0ea81a4960 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0ea6000700 1 -- 172.21.15.94:0/973519027 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e940043d0 con 0x7f0ea81a4960 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0ea6000700 1 -- 172.21.15.94:0/973519027 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94011b20 con 0x7f0ea81a4960 2024-04-25T14:53:13.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea81ab520 con 0x7f0ea81a4960 2024-04-25T14:53:13.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.044+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea81abab0 con 0x7f0ea81a4960 2024-04-25T14:53:13.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.046+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea80640a0 con 0x7f0ea81a4960 2024-04-25T14:53:13.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.046+0000 7f0ea6000700 1 -- 172.21.15.94:0/973519027 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0e94011cc0 con 0x7f0ea81a4960 2024-04-25T14:53:13.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.046+0000 7f0ea6000700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e90039ae0 0x7f0e9003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:13.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.046+0000 7f0ea6000700 1 -- 172.21.15.94:0/973519027 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0e9404c530 con 0x7f0ea81a4960 2024-04-25T14:53:13.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.047+0000 7f0ead800700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e90039ae0 0x7f0e9003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:13.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.047+0000 7f0ead800700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e90039ae0 0x7f0e9003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b5c0 tx=0x7f0e9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:13.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.052+0000 7f0ea6000700 1 -- 172.21.15.94:0/973519027 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e9400fe80 con 0x7f0ea81a4960 2024-04-25T14:53:13.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.281+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0ea81abd70 con 0x7f0ea81a4960 2024-04-25T14:53:13.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.282+0000 7f0ea6000700 1 -- 172.21.15.94:0/973519027 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f0e94018510 con 0x7f0ea81a4960 2024-04-25T14:53:13.283 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:13.283 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-04-25T14:53:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T14:53:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.286+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e90039ae0 msgr2=0x7f0e9003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.286+0000 7f0eb0200700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e90039ae0 0x7f0e9003bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00b5c0 tx=0x7f0e9c01a040 comp rx=0 tx=0).stop 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.286+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea81a4960 msgr2=0x7f0ea819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.286+0000 7f0eb0200700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea81a4960 0x7f0ea819e5e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400b780 tx=0x7f0e9400ba90 comp rx=0 tx=0).stop 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 shutdown_connections 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e90039ae0 0x7f0e9003bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ea819eb20 0x7f0ea819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ea8074de0 0x7f0ea81a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 --2- 172.21.15.94:0/973519027 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ea81a4960 0x7f0ea819e5e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:13.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 >> 172.21.15.94:0/973519027 conn(0x7f0ea80fdf80 msgr2=0x7f0ea80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:13.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 shutdown_connections 2024-04-25T14:53:13.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:13.287+0000 7f0eb0200700 1 -- 172.21.15.94:0/973519027 wait complete. 2024-04-25T14:53:13.697 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:13.698 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:13.699 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:13 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/973519027' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:13 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/973519027' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:13.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:13.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:13.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:13.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:13 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/973519027' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:14.109 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T14:53:14.109 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:53:14.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:53:14.238 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:53:14.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:53:14.361 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:53:14.361 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:53:14.392 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:53:14.392 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:53:14.459 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:53:14.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T14:53:14.490 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:53:14.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T14:53:14.558 INFO:tasks.cephadm:Adding mgr.x on smithi094 2024-04-25T14:53:14.559 INFO:tasks.cephadm:Adding mgr.y on smithi137 2024-04-25T14:53:14.559 INFO:tasks.cephadm:Adding mgr.z on smithi138 2024-04-25T14:53:14.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi094=x;smithi137=y;smithi138=z' 2024-04-25T14:53:14.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:14 smithi138 ceph-mon[32875]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:53:14.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:14 smithi138 ceph-mon[32875]: Reconfiguring daemon mon.a on smithi094 2024-04-25T14:53:14.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:14 smithi138 ceph-mon[32875]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:14.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:14 smithi094 ceph-mon[27862]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:53:14.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:14 smithi094 ceph-mon[27862]: Reconfiguring daemon mon.a on smithi094 2024-04-25T14:53:14.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:14 smithi094 ceph-mon[27862]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:14.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:14 smithi137 ceph-mon[32439]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T14:53:14.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:14 smithi137 ceph-mon[32439]: Reconfiguring daemon mon.a on smithi094 2024-04-25T14:53:14.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:14 smithi137 ceph-mon[32439]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:14.834 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 -- 172.21.15.138:0/479684366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81014e0 msgr2=0x7f28e8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 --2- 172.21.15.138:0/479684366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81014e0 0x7f28e8101900 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009910 tx=0x7f28dc009c20 comp rx=0 tx=0).stop 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 -- 172.21.15.138:0/479684366 shutdown_connections 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 --2- 172.21.15.138:0/479684366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81014e0 0x7f28e8101900 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 -- 172.21.15.138:0/479684366 >> 172.21.15.138:0/479684366 conn(0x7f28e80fca80 msgr2=0x7f28e80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 -- 172.21.15.138:0/479684366 shutdown_connections 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 -- 172.21.15.138:0/479684366 wait complete. 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 Processor -- start 2024-04-25T14:53:16.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.389+0000 7f28eee31700 1 -- start start 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28eee31700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e81014e0 0x7f28e8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28eee31700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81982e0 0x7f28e819d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28eee31700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e8198760 0x7f28e819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28eee31700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8107420 con 0x7f28e81982e0 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28eee31700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81072a0 con 0x7f28e8198760 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28eee31700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81075a0 con 0x7f28e81014e0 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e81014e0 0x7f28e8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e81014e0 0x7f28e8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35156/0 (socket says 172.21.15.138:35156) 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 -- 172.21.15.138:0/836913708 learned_addr learned my addr 172.21.15.138:0/836913708 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ed3ce700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e8198760 0x7f28e819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28e7fff700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81982e0 0x7f28e819d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 -- 172.21.15.138:0/836913708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e8198760 msgr2=0x7f28e819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:16.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e8198760 0x7f28e819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 -- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81982e0 msgr2=0x7f28e819d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81982e0 0x7f28e819d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ecbcd700 1 -- 172.21.15.138:0/836913708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e81a01c0 con 0x7f28e81014e0 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.390+0000 7f28ed3ce700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e8198760 0x7f28e819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.391+0000 7f28ecbcd700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e81014e0 0x7f28e8197da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0059d0 tx=0x7f28dc0050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.391+0000 7f28e5ffb700 1 -- 172.21.15.138:0/836913708 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc027070 con 0x7f28e81014e0 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.391+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e81a0450 con 0x7f28e81014e0 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.391+0000 7f28e5ffb700 1 -- 172.21.15.138:0/836913708 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28dc00be10 con 0x7f28e81014e0 2024-04-25T14:53:16.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.391+0000 7f28e5ffb700 1 -- 172.21.15.138:0/836913708 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28dc022e60 con 0x7f28e81014e0 2024-04-25T14:53:16.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.391+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e81a09b0 con 0x7f28e81014e0 2024-04-25T14:53:16.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.392+0000 7f28e5ffb700 1 -- 172.21.15.138:0/836913708 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f28dc02a070 con 0x7f28e81014e0 2024-04-25T14:53:16.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.392+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e80640a0 con 0x7f28e81014e0 2024-04-25T14:53:16.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.392+0000 7f28e5ffb700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f28d8039ae0 0x7f28d803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:16.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.392+0000 7f28e5ffb700 1 -- 172.21.15.138:0/836913708 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f28dc04dae0 con 0x7f28e81014e0 2024-04-25T14:53:16.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.392+0000 7f28e7fff700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f28d8039ae0 0x7f28d803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:16.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.393+0000 7f28e7fff700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f28d8039ae0 0x7f28d803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009850 tx=0x7f28d4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:16.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.397+0000 7f28e5ffb700 1 -- 172.21.15.138:0/836913708 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28dc013070 con 0x7f28e81014e0 2024-04-25T14:53:16.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.607+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}) -- 0x7f28e8062e80 con 0x7f28d8039ae0 2024-04-25T14:53:16.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.621+0000 7f28e5ffb700 1 -- 172.21.15.138:0/836913708 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f28e8062e80 con 0x7f28d8039ae0 2024-04-25T14:53:16.622 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mgr update... 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f28d8039ae0 msgr2=0x7f28d803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f28d8039ae0 0x7f28d803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009850 tx=0x7f28d4008040 comp rx=0 tx=0).stop 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e81014e0 msgr2=0x7f28e8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e81014e0 0x7f28e8197da0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f28dc0059d0 tx=0x7f28dc0050d0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 shutdown_connections 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f28d8039ae0 0x7f28d803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28e8198760 0x7f28e819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28e81014e0 0x7f28e8197da0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 --2- 172.21.15.138:0/836913708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28e81982e0 0x7f28e819d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 >> 172.21.15.138:0/836913708 conn(0x7f28e80fca80 msgr2=0x7f28e818ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 shutdown_connections 2024-04-25T14:53:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:53:16.623+0000 7f28eee31700 1 -- 172.21.15.138:0/836913708 wait complete. 2024-04-25T14:53:16.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:16.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:16.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:16.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:53:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:16 smithi138 ceph-mon[32875]: Reconfiguring daemon mon.b on smithi137 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:16 smithi094 ceph-mon[27862]: Reconfiguring daemon mon.b on smithi137 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: Reconfiguring mon.b (monmap changed)... 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:16 smithi137 ceph-mon[32439]: Reconfiguring daemon mon.b on smithi137 2024-04-25T14:53:17.203 DEBUG:teuthology.orchestra.run.smithi137:mgr.y> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T14:53:17.205 DEBUG:teuthology.orchestra.run.smithi138:mgr.z> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T14:53:17.208 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T14:53:17.208 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:53:17.208 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:53:17.267 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:53:17.267 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 14:52:18.245476994 +0000 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 14:46:25.682213085 +0000 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 14:46:25.682213085 +0000 2024-04-25T14:53:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 14:46:25.682213085 +0000 2024-04-25T14:53:17.323 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:53:17.399 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T14:53:17.399 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T14:53:17.399 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000386948 s, 1.3 MB/s 2024-04-25T14:53:17.402 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:53:17.456 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-04-25T14:53:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:53:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:17.502 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T14:53:17.503 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:17.503 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:17.503 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 14:52:18.313475876 +0000 2024-04-25T14:53:17.503 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 14:46:26.287203188 +0000 2024-04-25T14:53:17.503 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 14:46:26.287203188 +0000 2024-04-25T14:53:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 14:46:26.287203188 +0000 2024-04-25T14:53:17.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:53:17.578 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T14:53:17.578 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T14:53:17.578 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000269674 s, 1.9 MB/s 2024-04-25T14:53:17.582 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:53:17.638 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 14:52:18.382474742 +0000 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 14:46:26.882193454 +0000 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 14:46:26.882193454 +0000 2024-04-25T14:53:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 14:46:26.882193454 +0000 2024-04-25T14:53:17.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:53:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: Saving service mgr spec with placement smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T14:53:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:53:17.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:17.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:17.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:17.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: Reconfiguring daemon mon.c on smithi138 2024-04-25T14:53:17.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:17 smithi094 ceph-mon[27862]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T14:53:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T14:53:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000288424 s, 1.8 MB/s 2024-04-25T14:53:17.771 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:53:17.872 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-04-25T14:53:17.894 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:17.894 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: Saving service mgr spec with placement smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: Reconfiguring daemon mon.c on smithi138 2024-04-25T14:53:17.895 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:17 smithi138 ceph-mon[32875]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:53:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:17.924 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-25T14:53:17.924 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:17.924 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:17.924 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-04-25 14:52:18.457473509 +0000 2024-04-25T14:53:17.925 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-04-25 14:46:27.452184130 +0000 2024-04-25T14:53:17.925 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-04-25 14:46:27.452184130 +0000 2024-04-25T14:53:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-04-25 14:46:27.452184130 +0000 2024-04-25T14:53:17.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:53:17.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi137=y;smithi138=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:17.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: Saving service mgr spec with placement smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T14:53:17.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:17.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: Reconfiguring mon.c (monmap changed)... 2024-04-25T14:53:17.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T14:53:17.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T14:53:17.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:17.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: Reconfiguring daemon mon.c on smithi138 2024-04-25T14:53:17.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:17 smithi137 ceph-mon[32439]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:18.005 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-04-25T14:53:18.006 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-04-25T14:53:18.006 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000256749 s, 2.0 MB/s 2024-04-25T14:53:18.010 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:53:18.066 DEBUG:teuthology.orchestra.run.smithi137:> set -ex 2024-04-25T14:53:18.066 DEBUG:teuthology.orchestra.run.smithi137:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:53:18.083 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:53:18.083 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_1 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 14:53:03.847394659 +0000 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 14:45:20.820414135 +0000 2024-04-25T14:53:18.139 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 14:45:20.820414135 +0000 2024-04-25T14:53:18.140 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 14:45:20.820414135 +0000 2024-04-25T14:53:18.140 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:53:18.202 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T14:53:18.202 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T14:53:18.203 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000220727 s, 2.3 MB/s 2024-04-25T14:53:18.204 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:53:18.260 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_2 2024-04-25T14:53:18.315 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:53:18.315 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:18.315 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T14:53:18.316 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:18.316 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:18.316 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 14:53:03.847394659 +0000 2024-04-25T14:53:18.316 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 14:45:21.152408286 +0000 2024-04-25T14:53:18.316 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 14:45:21.152408286 +0000 2024-04-25T14:53:18.316 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 14:45:21.152408286 +0000 2024-04-25T14:53:18.316 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:53:18.378 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T14:53:18.378 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T14:53:18.378 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000245662 s, 2.1 MB/s 2024-04-25T14:53:18.381 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:53:18.437 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_3 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 14:53:03.847394659 +0000 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 14:45:21.518401837 +0000 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 14:45:21.518401837 +0000 2024-04-25T14:53:18.493 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 14:45:21.518401837 +0000 2024-04-25T14:53:18.493 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:53:18.556 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T14:53:18.556 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T14:53:18.556 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000245199 s, 2.1 MB/s 2024-04-25T14:53:18.558 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:53:18.614 DEBUG:teuthology.orchestra.run.smithi137:> stat /dev/vg_nvme/lv_4 2024-04-25T14:53:18.669 INFO:teuthology.orchestra.run.smithi137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout:Access: 2024-04-25 14:53:03.848394642 +0000 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout:Modify: 2024-04-25 14:45:21.860395812 +0000 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout:Change: 2024-04-25 14:45:21.860395812 +0000 2024-04-25T14:53:18.670 INFO:teuthology.orchestra.run.smithi137.stdout: Birth: 2024-04-25 14:45:21.860395812 +0000 2024-04-25T14:53:18.670 DEBUG:teuthology.orchestra.run.smithi137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:53:18.733 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records in 2024-04-25T14:53:18.733 INFO:teuthology.orchestra.run.smithi137.stderr:1+0 records out 2024-04-25T14:53:18.733 INFO:teuthology.orchestra.run.smithi137.stderr:512 bytes copied, 0.000196794 s, 2.6 MB/s 2024-04-25T14:53:18.734 DEBUG:teuthology.orchestra.run.smithi137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:53:18.793 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T14:53:18.794 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T14:53:18.809 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T14:53:18.809 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-25T14:53:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T14:53:18.864 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:18.864 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T14:53:18.864 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:18.865 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:18.865 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 14:52:54.065963014 +0000 2024-04-25T14:53:18.865 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 14:45:20.832153037 +0000 2024-04-25T14:53:18.865 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 14:45:20.832153037 +0000 2024-04-25T14:53:18.865 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 14:45:20.832153037 +0000 2024-04-25T14:53:18.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T14:53:18.928 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T14:53:18.928 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T14:53:18.928 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000213592 s, 2.4 MB/s 2024-04-25T14:53:18.929 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T14:53:18.984 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 14:52:54.065963014 +0000 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 14:45:21.176145644 +0000 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 14:45:21.176145644 +0000 2024-04-25T14:53:19.039 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 14:45:21.176145644 +0000 2024-04-25T14:53:19.039 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T14:53:19.102 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T14:53:19.103 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T14:53:19.103 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000182412 s, 2.8 MB/s 2024-04-25T14:53:19.104 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T14:53:19.161 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-25T14:53:19.216 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 14:52:54.066962993 +0000 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 14:45:21.522138207 +0000 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 14:45:21.522138207 +0000 2024-04-25T14:53:19.217 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 14:45:21.522138207 +0000 2024-04-25T14:53:19.217 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T14:53:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T14:53:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T14:53:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00019028 s, 2.7 MB/s 2024-04-25T14:53:19.279 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T14:53:19.335 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-25T14:53:19.389 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 14:52:54.066962993 +0000 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 14:45:21.881130491 +0000 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 14:45:21.881130491 +0000 2024-04-25T14:53:19.390 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 14:45:21.881130491 +0000 2024-04-25T14:53:19.390 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T14:53:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T14:53:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T14:53:19.452 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000187173 s, 2.7 MB/s 2024-04-25T14:53:19.453 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T14:53:19.508 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/vg_nvme/lv_4... 2024-04-25T14:53:19.509 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:53:19.928 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:53:19.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:19.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:53:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:19.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:53:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:19.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:53:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:20.653 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:20 smithi094 ceph-mon[27862]: Deploying daemon mgr.z on smithi138 2024-04-25T14:53:20.653 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:20 smithi094 ceph-mon[27862]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:20.707 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:20 smithi138 systemd[1]: Starting Ceph mgr.z for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:53:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:20 smithi138 ceph-mon[32875]: Deploying daemon mgr.z on smithi138 2024-04-25T14:53:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:20 smithi138 ceph-mon[32875]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:20 smithi137 ceph-mon[32439]: Deploying daemon mgr.z on smithi138 2024-04-25T14:53:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:20 smithi137 ceph-mon[32439]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:21.146 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:20 smithi138 podman[34715]: 2024-04-25 14:53:20.735324466 +0000 UTC m=+0.018272288 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:53:21.146 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:20 smithi138 podman[34715]: 2024-04-25 14:53:20.86748178 +0000 UTC m=+0.150429579 container create 2a448e3332f995b6ec4cc0ec710e5f0a7a92aa7a2a99c384bb0bf4459fdc6767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T14:53:21.146 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 podman[34715]: 2024-04-25 14:53:21.045608194 +0000 UTC m=+0.328555997 container init 2a448e3332f995b6ec4cc0ec710e5f0a7a92aa7a2a99c384bb0bf4459fdc6767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:53:21.146 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 podman[34715]: 2024-04-25 14:53:21.048965374 +0000 UTC m=+0.331913174 container start 2a448e3332f995b6ec4cc0ec710e5f0a7a92aa7a2a99c384bb0bf4459fdc6767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:53:21.146 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 ceph-mgr[34734]: -- 172.21.15.138:0/1049970132 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac61f0c5a0 con 0x55ac61ee1000 2024-04-25T14:53:21.147 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 bash[34715]: 2a448e3332f995b6ec4cc0ec710e5f0a7a92aa7a2a99c384bb0bf4459fdc6767 2024-04-25T14:53:21.456 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 systemd[1]: Started Ceph mgr.z for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:53:21.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:21.199+0000 7f7d404ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:53:21.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:21.384+0000 7f7d404ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:21.792 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:21.485+0000 7f7d404ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:53:22.206 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:21.791+0000 7f7d404ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:53:22.207 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:21.875+0000 7f7d404ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: Deploying daemon mgr.y on smithi137 2024-04-25T14:53:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:22 smithi137 ceph-mon[32439]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: Deploying daemon mgr.y on smithi137 2024-04-25T14:53:22.510 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:22 smithi138 ceph-mon[32875]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:22.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:22.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:22.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T14:53:22.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:22.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:22.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: Deploying daemon mgr.y on smithi137 2024-04-25T14:53:22.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:22 smithi094 ceph-mon[27862]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:22.710 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:53:22.710 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:53:22.710 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T14:53:22.711 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T14:53:22.711 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.036218 s, 290 MB/s 2024-04-25T14:53:22.711 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T14:53:23.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:23.113+0000 7f7d404ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:53:23.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:23.286+0000 7f7d404ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:23.457 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 systemd[1]: Starting Ceph mgr.y for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:53:23.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-04-25T14:53:23.726 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 podman[33813]: 2024-04-25 14:53:23.45661659 +0000 UTC m=+0.112146510 container create 10124ef5b4b5fdbbd1965f84277725c3a6b3dbccfff43cd36b5ada70a3385f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T14:53:23.726 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 podman[33813]: 2024-04-25 14:53:23.363233857 +0000 UTC m=+0.018763790 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:53:23.726 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 podman[33813]: 2024-04-25 14:53:23.659076703 +0000 UTC m=+0.314606623 container init 10124ef5b4b5fdbbd1965f84277725c3a6b3dbccfff43cd36b5ada70a3385f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T14:53:23.726 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 podman[33813]: 2024-04-25 14:53:23.66157469 +0000 UTC m=+0.317104610 container start 10124ef5b4b5fdbbd1965f84277725c3a6b3dbccfff43cd36b5ada70a3385f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T14:53:23.726 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 ceph-mgr[33832]: -- 172.21.15.137:0/4244480313 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df440485a0 con 0x55df4401d000 2024-04-25T14:53:23.726 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 bash[33813]: 10124ef5b4b5fdbbd1965f84277725c3a6b3dbccfff43cd36b5ada70a3385f65 2024-04-25T14:53:23.752 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:23.462+0000 7f7d404ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:53:23.752 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:23.751+0000 7f7d404ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:53:23.896 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:53:23.981 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 systemd[1]: Started Ceph mgr.y for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:53:23.982 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:23 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:23.815+0000 7f52f911a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T14:53:24.049 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:23.844+0000 7f7d404ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:53:24.049 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:24.048+0000 7f7d404ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:53:24.423 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:24.247+0000 7f7d404ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:24.424 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:24.002+0000 7f52f911a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:24.424 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:24.104+0000 7f52f911a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T14:53:24.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-mon[32875]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:24.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:24 smithi094 ceph-mon[27862]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:24.433+0000 7f52f911a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:24.520+0000 7f52f911a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-mon[32439]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:25.207 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:24.891+0000 7f7d404ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:53:25.629 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:25.324+0000 7f7d404ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:53:25.629 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:25.426+0000 7f7d404ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:53:25.629 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:25.628+0000 7f7d404ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:53:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 -- 172.21.15.94:0/3507002225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6988076a70 msgr2=0x7f6988074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:25.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 --2- 172.21.15.94:0/3507002225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6988076a70 0x7f6988074ed0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6978009990 tx=0x7f6978009ca0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 -- 172.21.15.94:0/3507002225 shutdown_connections 2024-04-25T14:53:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 --2- 172.21.15.94:0/3507002225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698810a8a0 0x7f698810cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 --2- 172.21.15.94:0/3507002225 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6988076a70 0x7f6988074ed0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 --2- 172.21.15.94:0/3507002225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69880754a0 0x7f6988075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 -- 172.21.15.94:0/3507002225 >> 172.21.15.94:0/3507002225 conn(0x7f69880fdf70 msgr2=0x7f69881003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 -- 172.21.15.94:0/3507002225 shutdown_connections 2024-04-25T14:53:25.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.823+0000 7f698d200700 1 -- 172.21.15.94:0/3507002225 wait complete. 2024-04-25T14:53:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.824+0000 7f698d200700 1 Processor -- start 2024-04-25T14:53:25.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.824+0000 7f698d200700 1 -- start start 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f698d200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69880754a0 0x7f698819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f698d200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6988076a70 0x7f698819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f698d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698810a8a0 0x7f69881a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f698d200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6988110690 con 0x7f6988076a70 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f698d200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6988110510 con 0x7f698810a8a0 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f698d200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6988110810 con 0x7f69880754a0 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69880754a0 0x7f698819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6986a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6988076a70 0x7f698819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69880754a0 0x7f698819ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35052/0 (socket says 172.21.15.94:35052) 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 -- 172.21.15.94:0/150620764 learned_addr learned my addr 172.21.15.94:0/150620764 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987e00700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698810a8a0 0x7f69881a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:25.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 -- 172.21.15.94:0/150620764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698810a8a0 msgr2=0x7f69881a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:25.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698810a8a0 0x7f69881a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 -- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6988076a70 msgr2=0x7f698819b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:25.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6988076a70 0x7f698819b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:25.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.825+0000 7f6987400700 1 -- 172.21.15.94:0/150620764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a2970 con 0x7f69880754a0 2024-04-25T14:53:25.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.826+0000 7f6987400700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69880754a0 0x7f698819ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f69780098e0 tx=0x7f69780057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:25.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.826+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978027070 con 0x7f69880754a0 2024-04-25T14:53:25.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.826+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6978022470 con 0x7f69880754a0 2024-04-25T14:53:25.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.826+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978017650 con 0x7f69880754a0 2024-04-25T14:53:25.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.826+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a2c00 con 0x7f69880754a0 2024-04-25T14:53:25.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.826+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a3000 con 0x7f69880754a0 2024-04-25T14:53:25.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.827+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6988069730 con 0x7f69880754a0 2024-04-25T14:53:25.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.828+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6978009d10 con 0x7f69880754a0 2024-04-25T14:53:25.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.828+0000 7f697fe00700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f696c039890 0x7f696c03bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:25.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.828+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f697804d4d0 con 0x7f69880754a0 2024-04-25T14:53:25.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.828+0000 7f6986a00700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f696c039890 0x7f696c03bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:25.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.829+0000 7f6986a00700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f696c039890 0x7f696c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69880767a0 tx=0x7f6974009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:25.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:25.833+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6978013070 con 0x7f69880754a0 2024-04-25T14:53:26.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:26.078+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6988062e80 con 0x7f696c039890 2024-04-25T14:53:26.116 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:25.781+0000 7f52f911a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T14:53:26.117 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:25.948+0000 7f52f911a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:26.117 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:26.116+0000 7f52f911a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T14:53:26.415 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:26.151+0000 7f7d404ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:53:26.416 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:26.242+0000 7f7d404ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:53:26.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:26.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:26.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:26.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:26.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:53:26.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:53:26.686 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:26.397+0000 7f52f911a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:26.489+0000 7f52f911a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:26.689+0000 7f52f911a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.690 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:53:26.691 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.706 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:26.483+0000 7f7d404ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:53:26.706 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:26.614+0000 7f7d404ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:53:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:26.982 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:26.883+0000 7f52f911a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T14:53:27.127 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:26.709+0000 7f7d404ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:53:27.128 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:26.824+0000 7f7d404ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:27.128 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:27.126+0000 7f7d404ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:53:27.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:27 smithi094 ceph-mon[27862]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:53:27.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:27 smithi094 ceph-mon[27862]: Reconfiguring daemon mgr.x on smithi094 2024-04-25T14:53:27.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:27 smithi094 ceph-mon[27862]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:27.535 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:27.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:27 smithi138 ceph-mon[32875]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:53:27.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:27 smithi138 ceph-mon[32875]: Reconfiguring daemon mgr.x on smithi094 2024-04-25T14:53:27.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:27 smithi138 ceph-mon[32875]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:27.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:27.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:27 smithi137 ceph-mon[32439]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T14:53:27.744 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:27 smithi137 ceph-mon[32439]: Reconfiguring daemon mgr.x on smithi094 2024-04-25T14:53:27.744 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:27 smithi137 ceph-mon[32439]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:27.744 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:27.744 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:27.495+0000 7f52f911a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T14:53:28.207 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:27.722+0000 7f7d404ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:53:28.207 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 14:53:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T14:53:27.938+0000 7f7d404ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:53:28.217 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:27.923+0000 7f52f911a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T14:53:28.217 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:28.019+0000 7f52f911a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T14:53:28.217 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:28.216+0000 7f52f911a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T14:53:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:53:28.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: Standby manager daemon z started 2024-04-25T14:53:28.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:53:28.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:53:28.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:53:28.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:28.700+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f697800fb10 con 0x7f69880754a0 2024-04-25T14:53:28.703 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:28.703 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.703 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.703 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:28.703 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:28.703 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:28.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:53:28.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: Standby manager daemon z started 2024-04-25T14:53:28.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:53:28.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:53:28.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:28 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:53:28.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:28.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: Standby manager daemon z started 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T14:53:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:28 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/2902087152' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:53:29.074 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:28.739+0000 7f52f911a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T14:53:29.074 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:28.830+0000 7f52f911a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T14:53:29.413 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:29.072+0000 7f52f911a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T14:53:29.413 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:29.203+0000 7f52f911a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T14:53:29.413 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:29.296+0000 7f52f911a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T14:53:29.413 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:29.411+0000 7f52f911a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T14:53:29.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:29 smithi138 ceph-mon[32875]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-25T14:53:29.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:53:29.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:29 smithi138 ceph-mon[32875]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:29 smithi094 ceph-mon[27862]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-25T14:53:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:53:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:29 smithi094 ceph-mon[27862]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:29.982 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:29.716+0000 7f52f911a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T14:53:29.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-mon[32439]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-25T14:53:29.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T14:53:29.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:29 smithi137 ceph-mon[32439]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:30.709 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:30.309+0000 7f52f911a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T14:53:30.709 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T14:53:30.523+0000 7f52f911a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T14:53:30.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:30.730+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mon.1 v2:172.21.15.138:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6978043650 con 0x7f69880754a0 2024-04-25T14:53:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:53:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: Standby manager daemon y started 2024-04-25T14:53:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:53:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:53:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:53:30.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2728399062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46fefd8-a368-4c2a-9038-54b8701077b7"}]: dispatch 2024-04-25T14:53:30.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2728399062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a46fefd8-a368-4c2a-9038-54b8701077b7"}]': finished 2024-04-25T14:53:30.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:53:30.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:30 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: Standby manager daemon y started 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2728399062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46fefd8-a368-4c2a-9038-54b8701077b7"}]: dispatch 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2728399062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a46fefd8-a368-4c2a-9038-54b8701077b7"}]': finished 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:53:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:30 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: Standby manager daemon y started 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/2888819537' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2728399062' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a46fefd8-a368-4c2a-9038-54b8701077b7"}]: dispatch 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2728399062' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a46fefd8-a368-4c2a-9038-54b8701077b7"}]': finished 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T14:53:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:30 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:31 smithi137 ceph-mon[32439]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-25T14:53:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:31 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:53:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:31 smithi137 ceph-mon[32439]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:31 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1853574568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:53:32.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:31 smithi138 ceph-mon[32875]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-25T14:53:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:31 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:53:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:31 smithi138 ceph-mon[32875]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:31 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1853574568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:53:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:31 smithi094 ceph-mon[27862]: mgrmap e15: x(active, since 89s), standbys: z, y 2024-04-25T14:53:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T14:53:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:31 smithi094 ceph-mon[27862]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:31 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1853574568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:53:34.694 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:34 smithi094 ceph-mon[27862]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:34.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:34 smithi138 ceph-mon[32875]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:34.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:34 smithi137 ceph-mon[32439]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:36.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:36 smithi138 ceph-mon[32875]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:36.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:36 smithi094 ceph-mon[27862]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:36.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:36 smithi137 ceph-mon[32439]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:38.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:38 smithi138 ceph-mon[32875]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:38.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:38 smithi094 ceph-mon[27862]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:38.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:38 smithi137 ceph-mon[32439]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:39.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:39 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:53:39.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:39 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:53:39.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:39 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T14:53:40.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:40 smithi138 ceph-mon[32875]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:40.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:40.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:40 smithi138 ceph-mon[32875]: Deploying daemon osd.0 on smithi094 2024-04-25T14:53:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:40 smithi094 ceph-mon[27862]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:40 smithi094 ceph-mon[27862]: Deploying daemon osd.0 on smithi094 2024-04-25T14:53:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:40 smithi137 ceph-mon[32439]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:40 smithi137 ceph-mon[32439]: Deploying daemon osd.0 on smithi094 2024-04-25T14:53:42.642 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:42 smithi094 ceph-mon[27862]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:42.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:42 smithi138 ceph-mon[32875]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:42.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:42 smithi137 ceph-mon[32439]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:44.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:44 smithi138 ceph-mon[32875]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:44 smithi094 ceph-mon[27862]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:44 smithi137 ceph-mon[32439]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:46.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:46 smithi138 ceph-mon[32875]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:46 smithi094 ceph-mon[27862]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:46 smithi137 ceph-mon[32439]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:48.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:48 smithi138 ceph-mon[32875]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:48.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:48.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:48.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:48.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:48 smithi094 ceph-mon[27862]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:48.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:48.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:48.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:48 smithi137 ceph-mon[32439]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:50.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:50 smithi094 ceph-mon[27862]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:50.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:50 smithi137 ceph-mon[32439]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:50.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:50 smithi138 ceph-mon[32875]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:51.935 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:51 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.935 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:51 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.935 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:51 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:51.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:51 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:51.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:51 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.936 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:51 smithi094 ceph-mon[27862]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:51.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:51 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:51 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:51 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:51.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:51 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:51.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:51 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:51 smithi138 ceph-mon[32875]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:51 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:51 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:51 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:51 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:51 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:51 smithi137 ceph-mon[32439]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:52.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.369+0000 7f697fe00700 1 -- 172.21.15.94:0/150620764 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6988062e80 con 0x7f696c039890 2024-04-25T14:53:52.371 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-04-25T14:53:52.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.372+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f696c039890 msgr2=0x7f696c03bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:52.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.372+0000 7f698d200700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f696c039890 0x7f696c03bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69880767a0 tx=0x7f6974009260 comp rx=0 tx=0).stop 2024-04-25T14:53:52.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69880754a0 msgr2=0x7f698819ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69880754a0 0x7f698819ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f69780098e0 tx=0x7f69780057c0 comp rx=0 tx=0).stop 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 shutdown_connections 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f696c039890 0x7f696c03bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698810a8a0 0x7f69881a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69880754a0 0x7f698819ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 --2- 172.21.15.94:0/150620764 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6988076a70 0x7f698819b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 >> 172.21.15.94:0/150620764 conn(0x7f69880fdf70 msgr2=0x7f69881067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 shutdown_connections 2024-04-25T14:53:52.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:53:52.373+0000 7f698d200700 1 -- 172.21.15.94:0/150620764 wait complete. 2024-04-25T14:53:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:52 smithi137 ceph-mon[32439]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:53:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:53.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:52 smithi094 ceph-mon[27862]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:53:53.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:53.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:53.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:53.104 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T14:53:53.108 INFO:tasks.cephadm:Deploying osd.1 on smithi137 with /dev/vg_nvme/lv_4... 2024-04-25T14:53:53.108 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:53:53.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:52 smithi138 ceph-mon[32875]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T14:53:53.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:53.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:53.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:53:53.350 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:53:53.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:53 smithi094 ceph-mon[27862]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:53:53.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:53 smithi094 ceph-mon[27862]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:53:53.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:53 smithi094 ceph-mon[27862]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:53:53.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:53 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:53.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:53 smithi094 ceph-mon[27862]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:53.963 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:53:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0[42494]: 2024-04-25T14:53:53.760+0000 7f0a3a400700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T14:53:53.964 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:53:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0[42494]: 2024-04-25T14:53:53.769+0000 7f0a35200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:53:54.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:53 smithi137 ceph-mon[32439]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:53:54.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:53 smithi137 ceph-mon[32439]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:53:54.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:53 smithi137 ceph-mon[32439]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:53:54.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:53 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:54.110 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:53 smithi137 ceph-mon[32439]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:54.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:53 smithi138 ceph-mon[32875]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T14:53:54.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:53 smithi138 ceph-mon[32875]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T14:53:54.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:53 smithi138 ceph-mon[32875]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:53:54.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:53 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:54.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:53 smithi138 ceph-mon[32875]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:54 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:53:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:54 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:53:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:54 smithi137 ceph-mon[32439]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:53:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:54 smithi137 ceph-mon[32439]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:53:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:54 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:54.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:54 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:55.035 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:54 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:53:55.035 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:54 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:53:55.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:54 smithi094 ceph-mon[27862]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:53:55.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:54 smithi094 ceph-mon[27862]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:53:55.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:54 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:55.036 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:54 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:55.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:54 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:53:55.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:54 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:53:55.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:54 smithi138 ceph-mon[32875]: from='osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:53:55.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:54 smithi138 ceph-mon[32875]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T14:53:55.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:54 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:55.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:54 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:55.287 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:53:55.287 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:53:55.287 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T14:53:55.287 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T14:53:55.288 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013037 s, 804 MB/s 2024-04-25T14:53:55.288 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T14:53:55.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:55 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:55.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:55 smithi137 ceph-mon[32439]: osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] boot 2024-04-25T14:53:55.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:55 smithi137 ceph-mon[32439]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:53:55.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:55 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:55.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:55 smithi137 ceph-mon[32439]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:56.019 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_4 2024-04-25T14:53:56.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:56.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:55 smithi138 ceph-mon[32875]: osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] boot 2024-04-25T14:53:56.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:55 smithi138 ceph-mon[32875]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:53:56.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:55 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:56.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:55 smithi138 ceph-mon[32875]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:56.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:55 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:56.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:55 smithi094 ceph-mon[27862]: osd.0 [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] boot 2024-04-25T14:53:56.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:55 smithi094 ceph-mon[27862]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T14:53:56.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:55 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T14:53:56.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:55 smithi094 ceph-mon[27862]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T14:53:56.256 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 -- 172.21.15.137:0/4160084145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c1035c0 msgr2=0x7f666c1059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 --2- 172.21.15.137:0/4160084145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c1035c0 0x7f666c1059b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f665c00b0a0 tx=0x7f665c00b3b0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 -- 172.21.15.137:0/4160084145 shutdown_connections 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 --2- 172.21.15.137:0/4160084145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c108820 0x7f666c10ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 --2- 172.21.15.137:0/4160084145 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c1035c0 0x7f666c1059b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 --2- 172.21.15.137:0/4160084145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c105ef0 0x7f666c1082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 -- 172.21.15.137:0/4160084145 >> 172.21.15.137:0/4160084145 conn(0x7f666c0fcf30 msgr2=0x7f666c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 -- 172.21.15.137:0/4160084145 shutdown_connections 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.695+0000 7f6672b09700 1 -- 172.21.15.137:0/4160084145 wait complete. 2024-04-25T14:53:57.698 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 Processor -- start 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 -- start start 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c1035c0 0x7f666c102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c105ef0 0x7f666c101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c108820 0x7f666c101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f666c10e5b0 con 0x7f666c1035c0 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f666c10e430 con 0x7f666c105ef0 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f6672b09700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f666c10e730 con 0x7f666c108820 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c105ef0 0x7f666c101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c105ef0 0x7f666c101570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33376/0 (socket says 172.21.15.137:33376) 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 -- 172.21.15.137:0/3069861768 learned_addr learned my addr 172.21.15.137:0/3069861768 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f66710a6700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c108820 0x7f666c101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 -- 172.21.15.137:0/3069861768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c108820 msgr2=0x7f666c101ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:57.699 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c108820 0x7f666c101ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 -- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c1035c0 msgr2=0x7f666c102f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c1035c0 0x7f666c102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 -- 172.21.15.137:0/3069861768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c102360 con 0x7f666c105ef0 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.696+0000 7f666bfff700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c105ef0 0x7f666c101570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f665400e9c0 tx=0x7f665400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.697+0000 7f6669ffb700 1 -- 172.21.15.137:0/3069861768 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665400cbd0 con 0x7f666c105ef0 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.697+0000 7f6669ffb700 1 -- 172.21.15.137:0/3069861768 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f665400cd70 con 0x7f666c105ef0 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.697+0000 7f6669ffb700 1 -- 172.21.15.137:0/3069861768 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654010430 con 0x7f666c105ef0 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.697+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c1a2db0 con 0x7f666c105ef0 2024-04-25T14:53:57.700 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.697+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c1a3290 con 0x7f666c105ef0 2024-04-25T14:53:57.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.698+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c069730 con 0x7f666c105ef0 2024-04-25T14:53:57.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.699+0000 7f6669ffb700 1 -- 172.21.15.137:0/3069861768 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6654004750 con 0x7f666c105ef0 2024-04-25T14:53:57.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.699+0000 7f6669ffb700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66580a3bb0 0x7f66580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:53:57.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.699+0000 7f6669ffb700 1 -- 172.21.15.137:0/3069861768 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f66540ced30 con 0x7f666c105ef0 2024-04-25T14:53:57.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.700+0000 7f66708a5700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66580a3bb0 0x7f66580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:53:57.701 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.701+0000 7f66708a5700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66580a3bb0 0x7f66580a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f665c009e60 tx=0x7f665c0060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:53:57.704 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.704+0000 7f6669ffb700 1 -- 172.21.15.137:0/3069861768 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66540d3070 con 0x7f666c105ef0 2024-04-25T14:53:57.910 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:53:57.909+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f666c062e80 con 0x7f66580a3bb0 2024-04-25T14:53:57.981 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:57 smithi137 ceph-mon[32439]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:53:57.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:57 smithi137 ceph-mon[32439]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:53:58.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:57 smithi138 ceph-mon[32875]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:53:58.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:57 smithi138 ceph-mon[32875]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:53:58.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:57 smithi094 ceph-mon[27862]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T14:53:58.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:57 smithi094 ceph-mon[27862]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:53:58.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:58.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi094 2024-04-25T14:53:58.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:53:58.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi094 to 17089M 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:58.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi094 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi094 to 17089M 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:59.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:59.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:53:59.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi094 2024-04-25T14:53:59.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:53:59.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:53:59.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:59.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:53:59.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi094 to 17089M 2024-04-25T14:53:59.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:53:59.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:53:59.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:53:59.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:00.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:53:59 smithi138 ceph-mon[32875]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:00.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:53:59 smithi094 ceph-mon[27862]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:00.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:53:59 smithi137 ceph-mon[32439]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:01.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:00 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/3831716760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]: dispatch 2024-04-25T14:54:01.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:00 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]: dispatch 2024-04-25T14:54:01.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:00 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]': finished 2024-04-25T14:54:01.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:00 smithi137 ceph-mon[32439]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:54:01.134 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:01.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:00 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/3831716760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]: dispatch 2024-04-25T14:54:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:00 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]: dispatch 2024-04-25T14:54:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:00 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]': finished 2024-04-25T14:54:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:00 smithi138 ceph-mon[32875]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:54:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:01.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:00 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/3831716760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]: dispatch 2024-04-25T14:54:01.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:00 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]: dispatch 2024-04-25T14:54:01.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:00 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "385d47c9-0cd9-424b-b924-411ae615e8ba"}]': finished 2024-04-25T14:54:01.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:00 smithi094 ceph-mon[27862]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T14:54:01.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:02.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:01 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/3630385330' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:02.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:01 smithi138 ceph-mon[32875]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:02.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:01 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/3630385330' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:02.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:01 smithi094 ceph-mon[27862]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:02.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:01 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/3630385330' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:02.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:01 smithi137 ceph-mon[32439]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:04.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:04 smithi138 ceph-mon[32875]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:04.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:04 smithi094 ceph-mon[27862]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:04.714 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:04 smithi137 ceph-mon[32439]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:06.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:06 smithi138 ceph-mon[32875]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:06.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:06 smithi094 ceph-mon[27862]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:06.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:06 smithi137 ceph-mon[32439]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:08 smithi137 ceph-mon[32439]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:54:08.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:08.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:08 smithi138 ceph-mon[32875]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:08.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:54:08.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:08 smithi094 ceph-mon[27862]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T14:54:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:09.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:09 smithi138 ceph-mon[32875]: Deploying daemon osd.1 on smithi137 2024-04-25T14:54:09.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:09 smithi137 ceph-mon[32439]: Deploying daemon osd.1 on smithi137 2024-04-25T14:54:09.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:09 smithi094 ceph-mon[27862]: Deploying daemon osd.1 on smithi137 2024-04-25T14:54:10.612 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:10 smithi137 ceph-mon[32439]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:10.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:10 smithi138 ceph-mon[32875]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:10.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:10 smithi094 ceph-mon[27862]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:12.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:12 smithi094 ceph-mon[27862]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:12.731 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:12 smithi137 ceph-mon[32439]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:12.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:12 smithi138 ceph-mon[32875]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:14.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:14.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:14.389 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:14.390 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:14 smithi137 ceph-mon[32439]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:14.456 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:14.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:14.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:14.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:14 smithi138 ceph-mon[32875]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:14.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:14.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:14.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:14.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:14 smithi094 ceph-mon[27862]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:16.452 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.451+0000 7f6669ffb700 1 -- 172.21.15.137:0/3069861768 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f666c062e80 con 0x7f66580a3bb0 2024-04-25T14:54:16.453 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 1 on host 'smithi137' 2024-04-25T14:54:16.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66580a3bb0 msgr2=0x7f66580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:16.454 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66580a3bb0 0x7f66580a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f665c009e60 tx=0x7f665c0060c0 comp rx=0 tx=0).stop 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c105ef0 msgr2=0x7f666c101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c105ef0 0x7f666c101570 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f665400e9c0 tx=0x7f665400ed80 comp rx=0 tx=0).stop 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 shutdown_connections 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66580a3bb0 0x7f66580a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f666c108820 0x7f666c101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f666c1035c0 0x7f666c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 --2- 172.21.15.137:0/3069861768 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f666c105ef0 0x7f666c101570 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 >> 172.21.15.137:0/3069861768 conn(0x7f666c0fcf30 msgr2=0x7f666c104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 shutdown_connections 2024-04-25T14:54:16.455 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:54:16.454+0000 7f6672b09700 1 -- 172.21.15.137:0/3069861768 wait complete. 2024-04-25T14:54:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:16 smithi138 ceph-mon[32875]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:16 smithi094 ceph-mon[27862]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:16 smithi137 ceph-mon[32439]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:17.056 DEBUG:teuthology.orchestra.run.smithi137:osd.1> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T14:54:17.059 INFO:tasks.cephadm:Deploying osd.2 on smithi138 with /dev/vg_nvme/lv_4... 2024-04-25T14:54:17.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T14:54:17.306 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:54:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:17 smithi138 ceph-mon[32875]: from='osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:54:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:17 smithi138 ceph-mon[32875]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:54:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:17 smithi137 ceph-mon[32439]: from='osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:54:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:17 smithi137 ceph-mon[32439]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:54:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:17 smithi094 ceph-mon[27862]: from='osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:54:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:17 smithi094 ceph-mon[27862]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T14:54:18.603 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:18 smithi138 ceph-mon[32875]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:18.603 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:18 smithi138 ceph-mon[32875]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:54:18.603 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:18 smithi138 ceph-mon[32875]: from='osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:54:18.603 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:18 smithi138 ceph-mon[32875]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:54:18.603 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:18.603 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:18 smithi138 ceph-mon[32875]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:54:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:18 smithi137 ceph-mon[32439]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:18 smithi137 ceph-mon[32439]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:54:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:18 smithi137 ceph-mon[32439]: from='osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:54:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:18 smithi137 ceph-mon[32439]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:54:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:18 smithi137 ceph-mon[32439]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:54:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:18 smithi094 ceph-mon[27862]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:18 smithi094 ceph-mon[27862]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T14:54:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:18 smithi094 ceph-mon[27862]: from='osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:54:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:18 smithi094 ceph-mon[27862]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T14:54:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:18 smithi094 ceph-mon[27862]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:54:19.131 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T14:54:19.131 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T14:54:19.131 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T14:54:19.131 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T14:54:19.131 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123069 s, 852 MB/s 2024-04-25T14:54:19.132 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T14:54:19.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:19 smithi138 ceph-mon[32875]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:54:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:19 smithi138 ceph-mon[32875]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:54:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:19 smithi138 ceph-mon[32875]: from='osd.1 ' entity='osd.1' 2024-04-25T14:54:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:19 smithi137 ceph-mon[32439]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:54:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:19 smithi137 ceph-mon[32439]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:54:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:19 smithi137 ceph-mon[32439]: from='osd.1 ' entity='osd.1' 2024-04-25T14:54:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:19.732 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1[37053]: 2024-04-25T14:54:19.422+0000 7f4247948700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T14:54:19.732 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1[37053]: 2024-04-25T14:54:19.544+0000 7f423ef32700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:54:19.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-04-25T14:54:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:19 smithi094 ceph-mon[27862]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:54:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:19 smithi094 ceph-mon[27862]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T14:54:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:19 smithi094 ceph-mon[27862]: from='osd.1 ' entity='osd.1' 2024-04-25T14:54:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:19.988 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:20.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:20.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:20.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:20.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:20.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:20.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:21.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 -- 172.21.15.138:0/4253608727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 msgr2=0x7ff7c0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 --2- 172.21.15.138:0/4253608727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0102ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009990 tx=0x7ff7b0009ca0 comp rx=0 tx=0).stop 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 -- 172.21.15.138:0/4253608727 shutdown_connections 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 --2- 172.21.15.138:0/4253608727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0104640 0x7ff7c010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 --2- 172.21.15.138:0/4253608727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0102ec0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 --2- 172.21.15.138:0/4253608727 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c0103ca0 0x7ff7c0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 -- 172.21.15.138:0/4253608727 >> 172.21.15.138:0/4253608727 conn(0x7ff7c00fe040 msgr2=0x7ff7c0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 -- 172.21.15.138:0/4253608727 shutdown_connections 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.433+0000 7ff7c557a700 1 -- 172.21.15.138:0/4253608727 wait complete. 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 Processor -- start 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 -- start start 2024-04-25T14:54:21.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0103ca0 0x7ff7c0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c0104640 0x7ff7c01a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010e610 con 0x7ff7c0103ca0 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010e490 con 0x7ff7c0104640 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7c557a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c010e790 con 0x7ff7c0102aa0 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33938/0 (socket says 172.21.15.138:33938) 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 -- 172.21.15.138:0/3509028758 learned_addr learned my addr 172.21.15.138:0/3509028758 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7bf7fe700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c0104640 0x7ff7c01a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 -- 172.21.15.138:0/3509028758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c0104640 msgr2=0x7ff7c01a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c0104640 0x7ff7c01a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 -- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0103ca0 msgr2=0x7ff7c0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:21.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0103ca0 0x7ff7c0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7beffd700 1 -- 172.21.15.138:0/3509028758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01a0960 con 0x7ff7c0102aa0 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.434+0000 7ff7bf7fe700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c0104640 0x7ff7c01a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.435+0000 7ff7beffd700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0198d10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00047b0 tx=0x7ff7b0003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.435+0000 7ff7a7fff700 1 -- 172.21.15.138:0/3509028758 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0027070 con 0x7ff7c0102aa0 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.435+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01a0bf0 con 0x7ff7c0102aa0 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.435+0000 7ff7a7fff700 1 -- 172.21.15.138:0/3509028758 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7b0022470 con 0x7ff7c0102aa0 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.435+0000 7ff7a7fff700 1 -- 172.21.15.138:0/3509028758 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000f6a0 con 0x7ff7c0102aa0 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.435+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01a1040 con 0x7ff7c0102aa0 2024-04-25T14:54:21.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.436+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0069770 con 0x7ff7c0102aa0 2024-04-25T14:54:21.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.437+0000 7ff7a7fff700 1 -- 172.21.15.138:0/3509028758 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff7b002a070 con 0x7ff7c0102aa0 2024-04-25T14:54:21.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.437+0000 7ff7a7fff700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff7a80a3bb0 0x7ff7a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:21.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.438+0000 7ff7a7fff700 1 -- 172.21.15.138:0/3509028758 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7ff7b00cfd10 con 0x7ff7c0102aa0 2024-04-25T14:54:21.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.438+0000 7ff7be7fc700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff7a80a3bb0 0x7ff7a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:21.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.439+0000 7ff7be7fc700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff7a80a3bb0 0x7ff7a80a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40097b0 tx=0x7ff7b4006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:21.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.441+0000 7ff7a7fff700 1 -- 172.21.15.138:0/3509028758 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7b0013070 con 0x7ff7c0102aa0 2024-04-25T14:54:21.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:21.646+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff7c0062e80 con 0x7ff7a80a3bb0 2024-04-25T14:54:21.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:21 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi137 2024-04-25T14:54:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:21 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-25T14:54:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:21 smithi138 ceph-mon[32875]: osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] boot 2024-04-25T14:54:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:21 smithi138 ceph-mon[32875]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:54:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:21 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:21 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi137 2024-04-25T14:54:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:21 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-25T14:54:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:21 smithi094 ceph-mon[27862]: osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] boot 2024-04-25T14:54:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:21 smithi094 ceph-mon[27862]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:54:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:21 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:21.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:21 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi137 2024-04-25T14:54:21.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:21 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi137 to 17088M 2024-04-25T14:54:21.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:21 smithi137 ceph-mon[32439]: osd.1 [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] boot 2024-04-25T14:54:21.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:21 smithi137 ceph-mon[32439]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T14:54:21.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:21 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T14:54:22.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:22 smithi138 ceph-mon[32875]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:22.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:22 smithi138 ceph-mon[32875]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:54:22.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:54:22.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:54:22.555 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:22 smithi094 ceph-mon[27862]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:22 smithi094 ceph-mon[27862]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:54:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:54:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:54:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:22 smithi137 ceph-mon[32439]: pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T14:54:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:22 smithi137 ceph-mon[32439]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T14:54:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:54:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:54:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:23.626 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:23 smithi138 ceph-mon[32875]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:23 smithi094 ceph-mon[27862]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:23.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:23 smithi137 ceph-mon[32439]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:24 smithi138 ceph-mon[32875]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:24 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/139863396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]: dispatch 2024-04-25T14:54:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:24 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]: dispatch 2024-04-25T14:54:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:24 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]': finished 2024-04-25T14:54:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:24 smithi138 ceph-mon[32875]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:54:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:24 smithi094 ceph-mon[27862]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:24 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/139863396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]: dispatch 2024-04-25T14:54:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:24 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]: dispatch 2024-04-25T14:54:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:24 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]': finished 2024-04-25T14:54:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:24 smithi094 ceph-mon[27862]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:54:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:24 smithi137 ceph-mon[32439]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:24 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/139863396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]: dispatch 2024-04-25T14:54:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:24 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]: dispatch 2024-04-25T14:54:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:24 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3318c983-93b2-43b5-ad3b-a833ac35c895"}]': finished 2024-04-25T14:54:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:24 smithi137 ceph-mon[32439]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T14:54:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:25.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:25 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/3899861143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:25.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:25 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/3899861143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:25.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:25 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/3899861143' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:26.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:26 smithi138 ceph-mon[32875]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:26.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:26 smithi094 ceph-mon[27862]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:26 smithi137 ceph-mon[32439]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:28 smithi138 ceph-mon[32875]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:28 smithi094 ceph-mon[27862]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:28 smithi137 ceph-mon[32439]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:30.894 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:30 smithi138 ceph-mon[32875]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:30 smithi094 ceph-mon[27862]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:30 smithi137 ceph-mon[32439]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:31.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:31 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:54:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:31 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:31.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:31 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:54:31.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:31 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:31.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T14:54:31.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:32.822 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:32 smithi138 ceph-mon[32875]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:32.822 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:32 smithi138 ceph-mon[32875]: Deploying daemon osd.2 on smithi138 2024-04-25T14:54:32.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:32 smithi094 ceph-mon[27862]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:32.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:32 smithi094 ceph-mon[27862]: Deploying daemon osd.2 on smithi138 2024-04-25T14:54:32.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:32 smithi137 ceph-mon[32439]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:32.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:32 smithi137 ceph-mon[32439]: Deploying daemon osd.2 on smithi138 2024-04-25T14:54:34.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:34 smithi138 ceph-mon[32875]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:34.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:34 smithi094 ceph-mon[27862]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:34 smithi137 ceph-mon[32439]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:36.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:36 smithi138 ceph-mon[32875]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:36.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:36 smithi094 ceph-mon[27862]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:36 smithi137 ceph-mon[32439]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:37 smithi137 ceph-mon[32439]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:38.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:38.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:38.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:38.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:37 smithi138 ceph-mon[32875]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:38.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:38.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:38.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:38.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:37 smithi094 ceph-mon[27862]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:40.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.277+0000 7ff7a7fff700 1 -- 172.21.15.138:0/3509028758 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff7c0062e80 con 0x7ff7a80a3bb0 2024-04-25T14:54:40.279 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 2 on host 'smithi138' 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff7a80a3bb0 msgr2=0x7ff7a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff7a80a3bb0 0x7ff7a80a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40097b0 tx=0x7ff7b4006d20 comp rx=0 tx=0).stop 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 msgr2=0x7ff7c0198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0198d10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00047b0 tx=0x7ff7b0003730 comp rx=0 tx=0).stop 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 shutdown_connections 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff7a80a3bb0 0x7ff7a80a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff7c0104640 0x7ff7c01a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7c0102aa0 0x7ff7c0198d10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 --2- 172.21.15.138:0/3509028758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7c0103ca0 0x7ff7c0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 >> 172.21.15.138:0/3509028758 conn(0x7ff7c00fe040 msgr2=0x7ff7c0107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 shutdown_connections 2024-04-25T14:54:40.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:54:40.280+0000 7ff7c557a700 1 -- 172.21.15.138:0/3509028758 wait complete. 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:40.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:54:40.864 DEBUG:teuthology.orchestra.run.smithi138:osd.2> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T14:54:40.866 INFO:tasks.cephadm:Deploying osd.3 on smithi094 with /dev/vg_nvme/lv_3... 2024-04-25T14:54:40.866 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:54:41.289 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:41.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:41 smithi094 ceph-mon[27862]: from='osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:54:41.568 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:41 smithi094 ceph-mon[27862]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:54:41.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:41 smithi138 ceph-mon[32875]: from='osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:54:41.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:41 smithi138 ceph-mon[32875]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:54:41.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:41 smithi137 ceph-mon[32439]: from='osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:54:41.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:41 smithi137 ceph-mon[32439]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T14:54:42.616 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:42 smithi094 ceph-mon[27862]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:42.617 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:42 smithi094 ceph-mon[27862]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:54:42.617 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:42 smithi094 ceph-mon[27862]: from='osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:54:42.617 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:42 smithi094 ceph-mon[27862]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:54:42.617 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:42.617 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:42 smithi094 ceph-mon[27862]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:54:42.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:42 smithi138 ceph-mon[32875]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:42.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:42 smithi138 ceph-mon[32875]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:54:42.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:42 smithi138 ceph-mon[32875]: from='osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:54:42.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:42 smithi138 ceph-mon[32875]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:54:42.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:42.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:42 smithi138 ceph-mon[32875]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:54:42.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:42 smithi137 ceph-mon[32439]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:42.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:42 smithi137 ceph-mon[32439]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T14:54:42.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:42 smithi137 ceph-mon[32439]: from='osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:54:42.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:42 smithi137 ceph-mon[32439]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T14:54:42.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:42.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:42 smithi137 ceph-mon[32439]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:54:43.422 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T14:54:43.335+0000 7f80dcd5f700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T14:54:43.422 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T14:54:43.420+0000 7f80d4b4a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:54:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:43 smithi094 ceph-mon[27862]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:54:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:43 smithi094 ceph-mon[27862]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:54:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:43 smithi094 ceph-mon[27862]: from='osd.2 ' entity='osd.2' 2024-04-25T14:54:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:43 smithi138 ceph-mon[32875]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:54:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:43 smithi138 ceph-mon[32875]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:54:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:43 smithi138 ceph-mon[32875]: from='osd.2 ' entity='osd.2' 2024-04-25T14:54:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:43 smithi137 ceph-mon[32439]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:54:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:43 smithi137 ceph-mon[32439]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T14:54:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:43 smithi137 ceph-mon[32439]: from='osd.2 ' entity='osd.2' 2024-04-25T14:54:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:54:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:54:44.075 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T14:54:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T14:54:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0339944 s, 308 MB/s 2024-04-25T14:54:44.075 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T14:54:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:54:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:54:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] boot 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:54:44.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:54:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:54:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] boot 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:54:44.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:54:44.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:44.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: osd.2 [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] boot 2024-04-25T14:54:44.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T14:54:44.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T14:54:44.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-04-25T14:54:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:54:45.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:45 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi138 2024-04-25T14:54:45.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:45 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi138 to 17088M 2024-04-25T14:54:45.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:45 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi138 2024-04-25T14:54:45.709 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:45 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi138 to 17088M 2024-04-25T14:54:45.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:45 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi138 2024-04-25T14:54:45.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:45 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi138 to 17088M 2024-04-25T14:54:46.536 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:46 smithi094 ceph-mon[27862]: pgmap v90: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:46.536 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:46 smithi094 ceph-mon[27862]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:54:46.536 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:54:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:46 smithi137 ceph-mon[32439]: pgmap v90: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:46 smithi137 ceph-mon[32439]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:54:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:54:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:46 smithi138 ceph-mon[32875]: pgmap v90: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:46 smithi138 ceph-mon[32875]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T14:54:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:54:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.382+0000 7f25a6a00700 1 -- 172.21.15.94:0/2549193818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0104620 msgr2=0x7f25a010aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.382+0000 7f25a6a00700 1 --2- 172.21.15.94:0/2549193818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0104620 0x7f25a010aed0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f259800b0a0 tx=0x7f259800b3b0 comp rx=0 tx=0).stop 2024-04-25T14:54:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.382+0000 7f25a6a00700 1 -- 172.21.15.94:0/2549193818 shutdown_connections 2024-04-25T14:54:47.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.382+0000 7f25a6a00700 1 --2- 172.21.15.94:0/2549193818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0104620 0x7f25a010aed0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.382+0000 7f25a6a00700 1 --2- 172.21.15.94:0/2549193818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a0100ca0 0x7f25a01010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.382+0000 7f25a6a00700 1 --2- 172.21.15.94:0/2549193818 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25a0103c80 0x7f25a01040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.382+0000 7f25a6a00700 1 -- 172.21.15.94:0/2549193818 >> 172.21.15.94:0/2549193818 conn(0x7f25a00fc600 msgr2=0x7f25a00fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:54:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.383+0000 7f25a6a00700 1 -- 172.21.15.94:0/2549193818 shutdown_connections 2024-04-25T14:54:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.383+0000 7f25a6a00700 1 -- 172.21.15.94:0/2549193818 wait complete. 2024-04-25T14:54:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.384+0000 7f25a6a00700 1 Processor -- start 2024-04-25T14:54:47.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.384+0000 7f25a6a00700 1 -- start start 2024-04-25T14:54:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25a0100ca0 0x7f25a019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0103c80 0x7f25a019b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a6a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a0104620 0x7f25a01a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:47.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a6a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f25a010e450 con 0x7f25a0103c80 2024-04-25T14:54:47.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a6a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f25a010e2d0 con 0x7f25a0100ca0 2024-04-25T14:54:47.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a6a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25a010e5d0 con 0x7f25a0104620 2024-04-25T14:54:47.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a4c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0103c80 0x7f25a019b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:47.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a4c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0103c80 0x7f25a019b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60084/0 (socket says 172.21.15.94:60084) 2024-04-25T14:54:47.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a4c00700 1 -- 172.21.15.94:0/1832656234 learned_addr learned my addr 172.21.15.94:0/1832656234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:54:47.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a5600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25a0100ca0 0x7f25a019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:47.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.385+0000 7f25a6000700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a0104620 0x7f25a01a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f25a6000700 1 -- 172.21.15.94:0/1832656234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25a0100ca0 msgr2=0x7f25a019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f25a6000700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25a0100ca0 0x7f25a019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f25a6000700 1 -- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0103c80 msgr2=0x7f25a019b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f25a6000700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0103c80 0x7f25a019b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f25a6000700 1 -- 172.21.15.94:0/1832656234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a01a28b0 con 0x7f25a0104620 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f25a6000700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a0104620 0x7f25a01a21e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f259800b070 tx=0x7f2598004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598021070 con 0x7f25a0104620 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a01a2b40 con 0x7f25a0104620 2024-04-25T14:54:47.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2598007c40 con 0x7f25a0104620 2024-04-25T14:54:47.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.386+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980052c0 con 0x7f25a0104620 2024-04-25T14:54:47.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.387+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a01a30d0 con 0x7f25a0104620 2024-04-25T14:54:47.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.388+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a0069730 con 0x7f25a0104620 2024-04-25T14:54:47.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.390+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2598024070 con 0x7f25a0104620 2024-04-25T14:54:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.391+0000 7f259e000700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25880a3c30 0x7f25880a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:54:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.391+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f259800f020 con 0x7f25a0104620 2024-04-25T14:54:47.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.391+0000 7f25a5600700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25880a3c30 0x7f25880a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:54:47.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.392+0000 7f25a5600700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25880a3c30 0x7f25880a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25a019c140 tx=0x7f2594009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:54:47.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.394+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f259802b020 con 0x7f25a0104620 2024-04-25T14:54:47.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:47.658+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f25a01a3360 con 0x7f25880a3c30 2024-04-25T14:54:47.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:47 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:54:47.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:47 smithi137 ceph-mon[32439]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:54:47.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:47 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:54:47.760 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:47 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:54:47.760 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:47 smithi094 ceph-mon[27862]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:54:47.760 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:47 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:54:47.760 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47161]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:54:47.760 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47161]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:47.760 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47161]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:47.760 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47161]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:47.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:47 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T14:54:47.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:47 smithi138 ceph-mon[32875]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T14:54:47.873 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:47 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T14:54:48.017 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47231]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:54:48.017 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.017 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47231]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.018 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47231]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.018 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:54:48.018 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.018 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.018 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 14:54:47 smithi094 sudo[47187]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41164]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41164]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41164]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41167]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.081 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 14:54:47 smithi137 sudo[41167]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42097]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42097]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42097]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42097]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.207 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 14:54:47 smithi138 sudo[42100]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42103]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:54:48.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42103]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42103]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42103]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42106]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:54:48.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42106]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42106]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 sudo[42106]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 sudo[47234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:54:48.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 sudo[47234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 sudo[47234]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 sudo[47234]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41170]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41170]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41170]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41170]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41173]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41173]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41173]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T14:54:48.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 sudo[41173]: pam_unix(sudo:session): session closed for user root 2024-04-25T14:54:48.761 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:48.761 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:54:48.761 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:54:48.761 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:54:48.761 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:54:48.761 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.763 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:48 smithi094 ceph-mon[27862]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.813 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.814 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:48 smithi138 ceph-mon[32875]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: pgmap v93: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T14:54:48.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:48 smithi137 ceph-mon[32439]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T14:54:49.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:54:49.515+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f259809a760 con 0x7f25a0104620 2024-04-25T14:54:49.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:49 smithi094 ceph-mon[27862]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:49.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:49 smithi094 ceph-mon[27862]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:54:49.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:49 smithi138 ceph-mon[32875]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:49.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:49 smithi138 ceph-mon[32875]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:54:49.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:49 smithi137 ceph-mon[32439]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:54:49.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:49 smithi137 ceph-mon[32439]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T14:54:50.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:50 smithi094 ceph-mon[27862]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:50.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:50 smithi094 ceph-mon[27862]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T14:54:50.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:50 smithi094 ceph-mon[27862]: Cluster is now healthy 2024-04-25T14:54:50.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:50 smithi094 ceph-mon[27862]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:54:50.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:50 smithi138 ceph-mon[32875]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:50 smithi138 ceph-mon[32875]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T14:54:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:50 smithi138 ceph-mon[32875]: Cluster is now healthy 2024-04-25T14:54:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:50 smithi138 ceph-mon[32875]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:54:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:50 smithi137 ceph-mon[32439]: pgmap v96: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:50 smithi137 ceph-mon[32439]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T14:54:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:50 smithi137 ceph-mon[32439]: Cluster is now healthy 2024-04-25T14:54:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:50 smithi137 ceph-mon[32439]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T14:54:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:52 smithi138 ceph-mon[32875]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:52 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3897634347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afc1aca-e4d4-405f-a3e6-f80504727eec"}]: dispatch 2024-04-25T14:54:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:52 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3897634347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afc1aca-e4d4-405f-a3e6-f80504727eec"}]': finished 2024-04-25T14:54:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:52 smithi138 ceph-mon[32875]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:54:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:54:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:52 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1208621725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:52 smithi094 ceph-mon[27862]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:52 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3897634347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afc1aca-e4d4-405f-a3e6-f80504727eec"}]: dispatch 2024-04-25T14:54:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:52 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3897634347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afc1aca-e4d4-405f-a3e6-f80504727eec"}]': finished 2024-04-25T14:54:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:52 smithi094 ceph-mon[27862]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:54:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:54:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:52 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1208621725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:52 smithi137 ceph-mon[32439]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:52 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3897634347' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0afc1aca-e4d4-405f-a3e6-f80504727eec"}]: dispatch 2024-04-25T14:54:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:52 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3897634347' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0afc1aca-e4d4-405f-a3e6-f80504727eec"}]': finished 2024-04-25T14:54:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:52 smithi137 ceph-mon[32439]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T14:54:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:54:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:52 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1208621725' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:54:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:54 smithi138 ceph-mon[32875]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:54.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:54 smithi094 ceph-mon[27862]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:54 smithi137 ceph-mon[32439]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:56.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:56 smithi094 ceph-mon[27862]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:56 smithi138 ceph-mon[32875]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:56 smithi137 ceph-mon[32439]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:54:58 smithi094 ceph-mon[27862]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:58.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:54:58 smithi138 ceph-mon[32875]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:54:58.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:54:58 smithi137 ceph-mon[32439]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:00 smithi138 ceph-mon[32875]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:55:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:00 smithi094 ceph-mon[27862]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:55:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:00.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:00 smithi137 ceph-mon[32439]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:00.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T14:55:00.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:01.595 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:01 smithi094 ceph-mon[27862]: Deploying daemon osd.3 on smithi094 2024-04-25T14:55:01.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:01 smithi137 ceph-mon[32439]: Deploying daemon osd.3 on smithi094 2024-04-25T14:55:01.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:01 smithi138 ceph-mon[32875]: Deploying daemon osd.3 on smithi094 2024-04-25T14:55:02.870 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:02 smithi094 ceph-mon[27862]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:02 smithi138 ceph-mon[32875]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:02.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:02 smithi137 ceph-mon[32439]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:04.876 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:04 smithi094 ceph-mon[27862]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:04.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:04 smithi138 ceph-mon[32875]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:04 smithi137 ceph-mon[32439]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:06.799 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:06 smithi094 ceph-mon[27862]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:06 smithi138 ceph-mon[32875]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:06 smithi137 ceph-mon[32439]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:08 smithi138 ceph-mon[32875]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:08.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:08 smithi094 ceph-mon[27862]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:08.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:08.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:08.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:08 smithi137 ceph-mon[32439]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:10 smithi138 ceph-mon[32875]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:10 smithi094 ceph-mon[27862]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:10 smithi137 ceph-mon[32439]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:12 smithi138 ceph-mon[32875]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:12 smithi138 ceph-mon[32875]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:55:12.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:12 smithi094 ceph-mon[27862]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:12.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:12.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:12.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:12 smithi094 ceph-mon[27862]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:55:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:12 smithi137 ceph-mon[32439]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:12 smithi137 ceph-mon[32439]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T14:55:13.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.110+0000 7f259e000700 1 -- 172.21.15.94:0/1832656234 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25a01a3360 con 0x7f25880a3c30 2024-04-25T14:55:13.112 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 3 on host 'smithi094' 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.113+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25880a3c30 msgr2=0x7f25880a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.113+0000 7f25a6a00700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25880a3c30 0x7f25880a60f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f25a019c140 tx=0x7f2594009260 comp rx=0 tx=0).stop 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.113+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a0104620 msgr2=0x7f25a01a21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.113+0000 7f25a6a00700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a0104620 0x7f25a01a21e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f259800b070 tx=0x7f2598004d70 comp rx=0 tx=0).stop 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 shutdown_connections 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25880a3c30 0x7f25880a60f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25a0104620 0x7f25a01a21e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f25a0100ca0 0x7f25a019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:13.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 --2- 172.21.15.94:0/1832656234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f25a0103c80 0x7f25a019b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:13.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 >> 172.21.15.94:0/1832656234 conn(0x7f25a00fc600 msgr2=0x7f25a0101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:13.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 shutdown_connections 2024-04-25T14:55:13.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:55:13.114+0000 7f25a6a00700 1 -- 172.21.15.94:0/1832656234 wait complete. 2024-04-25T14:55:13.916 DEBUG:teuthology.orchestra.run.smithi094:osd.3> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T14:55:13.922 INFO:tasks.cephadm:Deploying osd.4 on smithi137 with /dev/vg_nvme/lv_3... 2024-04-25T14:55:13.922 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:55:14.135 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:55:14.135 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:55:14.135 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:55:14.135 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:14.135 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:14.135 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:14.135 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:14.136 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:13 smithi094 ceph-mon[27862]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:14.167 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:13 smithi138 ceph-mon[32875]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:13 smithi137 ceph-mon[32439]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-25T14:55:15.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:14 smithi094 ceph-mon[27862]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:55:15.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:14 smithi094 ceph-mon[27862]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:55:15.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:14 smithi094 ceph-mon[27862]: osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] boot 2024-04-25T14:55:15.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:14 smithi094 ceph-mon[27862]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:55:15.034 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:14 smithi138 ceph-mon[32875]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:55:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:14 smithi138 ceph-mon[32875]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:55:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:14 smithi138 ceph-mon[32875]: osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] boot 2024-04-25T14:55:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:14 smithi138 ceph-mon[32875]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:55:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:14 smithi137 ceph-mon[32439]: from='osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:55:15.265 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:14 smithi137 ceph-mon[32439]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T14:55:15.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:15.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:14 smithi137 ceph-mon[32439]: osd.3 [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] boot 2024-04-25T14:55:15.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:14 smithi137 ceph-mon[32439]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T14:55:15.266 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T14:55:16.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:15 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:55:16.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:15 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:55:16.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:15 smithi138 ceph-mon[32875]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:16.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:15 smithi138 ceph-mon[32875]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:55:16.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:15 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:55:16.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:15 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:55:16.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:15 smithi094 ceph-mon[27862]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:16.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:15 smithi094 ceph-mon[27862]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:55:16.225 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:55:16.226 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:55:16.226 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T14:55:16.226 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T14:55:16.226 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0120714 s, 869 MB/s 2024-04-25T14:55:16.226 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T14:55:16.226 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:15 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:55:16.226 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:15 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:55:16.226 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:15 smithi137 ceph-mon[32439]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:16.226 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:15 smithi137 ceph-mon[32439]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T14:55:16.893 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_3 2024-04-25T14:55:17.130 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:55:18.178 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:17 smithi137 ceph-mon[32439]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:55:18.178 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:17 smithi137 ceph-mon[32439]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:18.183 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:17 smithi094 ceph-mon[27862]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:55:18.183 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:17 smithi094 ceph-mon[27862]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:18.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:17 smithi138 ceph-mon[32875]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T14:55:18.206 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:17 smithi138 ceph-mon[32875]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:18.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 -- 172.21.15.137:0/1632858561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088102bb0 msgr2=0x7f4088103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 --2- 172.21.15.137:0/1632858561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088102bb0 0x7f4088103010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f40700099c0 tx=0x7f4070009cd0 comp rx=0 tx=0).stop 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 -- 172.21.15.137:0/1632858561 shutdown_connections 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 --2- 172.21.15.137:0/1632858561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088103550 0x7f4088109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 --2- 172.21.15.137:0/1632858561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f40881019b0 0x7f4088101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 --2- 172.21.15.137:0/1632858561 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088102bb0 0x7f4088103010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 -- 172.21.15.137:0/1632858561 >> 172.21.15.137:0/1632858561 conn(0x7f40880fcf70 msgr2=0x7f40880ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 -- 172.21.15.137:0/1632858561 shutdown_connections 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 -- 172.21.15.137:0/1632858561 wait complete. 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 Processor -- start 2024-04-25T14:55:18.662 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.659+0000 7f408dfb3700 1 -- start start 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f408dfb3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40881019b0 0x7f4088105940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f408dfb3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4088102bb0 0x7f4088103f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f408dfb3700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088103550 0x7f40881044d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f408dfb3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f408810d4a0 con 0x7f40881019b0 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f408dfb3700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f408810d320 con 0x7f4088103550 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f408dfb3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f408810d620 con 0x7f4088102bb0 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088103550 0x7f40881044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088103550 0x7f40881044d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:33442/0 (socket says 172.21.15.137:33442) 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 -- 172.21.15.137:0/646623707 learned_addr learned my addr 172.21.15.137:0/646623707 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f407effd700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4088102bb0 0x7f4088103f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 -- 172.21.15.137:0/646623707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4088102bb0 msgr2=0x7f4088103f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4088102bb0 0x7f4088103f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:18.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 -- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40881019b0 msgr2=0x7f4088105940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40881019b0 0x7f4088105940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 -- 172.21.15.137:0/646623707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4088104d80 con 0x7f4088103550 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f4087fff700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088103550 0x7f40881044d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f408000b970 tx=0x7f408000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f40857fa700 1 -- 172.21.15.137:0/646623707 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080019070 con 0x7f4088103550 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.660+0000 7f40857fa700 1 -- 172.21.15.137:0/646623707 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408000cde0 con 0x7f4088103550 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.661+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40881a2e10 con 0x7f4088103550 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.661+0000 7f40857fa700 1 -- 172.21.15.137:0/646623707 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4080007370 con 0x7f4088103550 2024-04-25T14:55:18.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.661+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40881a3270 con 0x7f4088103550 2024-04-25T14:55:18.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.662+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4088069730 con 0x7f4088103550 2024-04-25T14:55:18.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.663+0000 7f40857fa700 1 -- 172.21.15.137:0/646623707 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f408001c070 con 0x7f4088103550 2024-04-25T14:55:18.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.663+0000 7f40857fa700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f40740a3c40 0x7f40740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:18.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.663+0000 7f40857fa700 1 -- 172.21.15.137:0/646623707 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f40800ce560 con 0x7f4088103550 2024-04-25T14:55:18.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.664+0000 7f40877fe700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f40740a3c40 0x7f40740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:18.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.665+0000 7f40877fe700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f40740a3c40 0x7f40740a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4088102a10 tx=0x7f4078009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:18.669 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.668+0000 7f40857fa700 1 -- 172.21.15.137:0/646623707 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f408009c160 con 0x7f4088103550 2024-04-25T14:55:18.877 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:18.876+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f408806b8d0 con 0x7f40740a3c40 2024-04-25T14:55:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:55:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi094 2024-04-25T14:55:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T14:55:19.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:19.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:19.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:19.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:19.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:19.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:55:19.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi094 2024-04-25T14:55:19.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:19.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:19.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T14:55:19.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi094 2024-04-25T14:55:19.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:19.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi094 to 8544M 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:19.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:20.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:19 smithi138 ceph-mon[32875]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:20.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:19 smithi138 ceph-mon[32875]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:55:20.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:19 smithi138 ceph-mon[32875]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:20.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:19 smithi094 ceph-mon[27862]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:20.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:19 smithi094 ceph-mon[27862]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:55:20.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:19 smithi094 ceph-mon[27862]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:20.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:19 smithi137 ceph-mon[32439]: from='client.24227 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:20.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:19 smithi137 ceph-mon[32439]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T14:55:20.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:19 smithi137 ceph-mon[32439]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:22 smithi138 ceph-mon[32875]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:22 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/282970298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]: dispatch 2024-04-25T14:55:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:22 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]: dispatch 2024-04-25T14:55:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:22 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]': finished 2024-04-25T14:55:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:22 smithi138 ceph-mon[32875]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:55:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:22 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/718479366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:22 smithi094 ceph-mon[27862]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:22 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/282970298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]: dispatch 2024-04-25T14:55:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:22 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]: dispatch 2024-04-25T14:55:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:22 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]': finished 2024-04-25T14:55:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:22 smithi094 ceph-mon[27862]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:55:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:22 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/718479366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:22 smithi137 ceph-mon[32439]: pgmap v120: 1 pgs: 1 remapped+peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:22 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/282970298' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]: dispatch 2024-04-25T14:55:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:22 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]: dispatch 2024-04-25T14:55:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:22 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8898932-385f-4e12-a84d-074556811d16"}]': finished 2024-04-25T14:55:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:22 smithi137 ceph-mon[32439]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T14:55:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:22 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/718479366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:24 smithi094 ceph-mon[27862]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:55:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:24 smithi137 ceph-mon[32439]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:55:24.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:24 smithi138 ceph-mon[32875]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:55:26.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:26 smithi137 ceph-mon[32439]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T14:55:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:26 smithi138 ceph-mon[32875]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T14:55:26.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:26 smithi094 ceph-mon[27862]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-25T14:55:28.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:28 smithi137 ceph-mon[32439]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T14:55:28.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:28 smithi138 ceph-mon[32875]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T14:55:28.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:28 smithi094 ceph-mon[27862]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-25T14:55:29.757 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:55:29.758 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:29.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:55:29.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T14:55:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:30.747 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:30 smithi137 ceph-mon[32439]: Deploying daemon osd.4 on smithi137 2024-04-25T14:55:30.748 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:30 smithi137 ceph-mon[32439]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:55:30.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:30 smithi138 ceph-mon[32875]: Deploying daemon osd.4 on smithi137 2024-04-25T14:55:30.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:30 smithi138 ceph-mon[32875]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:55:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:30 smithi094 ceph-mon[27862]: Deploying daemon osd.4 on smithi137 2024-04-25T14:55:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:30 smithi094 ceph-mon[27862]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:55:32.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:32 smithi137 ceph-mon[32439]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:55:32.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:32 smithi138 ceph-mon[32875]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:55:32.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:32 smithi094 ceph-mon[27862]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:55:34.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:34 smithi137 ceph-mon[32439]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:34.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:34.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:34.781 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:34 smithi138 ceph-mon[32875]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:34.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:34 smithi094 ceph-mon[27862]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:34.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:34.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:34.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:36.707 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:36 smithi137 ceph-mon[32439]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:36.759 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:36 smithi138 ceph-mon[32875]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:36.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:36 smithi094 ceph-mon[27862]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:38.069 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.068+0000 7f40857fa700 1 -- 172.21.15.137:0/646623707 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f408806b8d0 con 0x7f40740a3c40 2024-04-25T14:55:38.069 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 4 on host 'smithi137' 2024-04-25T14:55:38.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f40740a3c40 msgr2=0x7f40740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:38.071 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f40740a3c40 0x7f40740a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4088102a10 tx=0x7f4078009470 comp rx=0 tx=0).stop 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088103550 msgr2=0x7f40881044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088103550 0x7f40881044d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f408000b970 tx=0x7f408000bc80 comp rx=0 tx=0).stop 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 shutdown_connections 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f40740a3c40 0x7f40740a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4088103550 0x7f40881044d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f40881019b0 0x7f4088105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 --2- 172.21.15.137:0/646623707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4088102bb0 0x7f4088103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 >> 172.21.15.137:0/646623707 conn(0x7f40880fcf70 msgr2=0x7f408806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 shutdown_connections 2024-04-25T14:55:38.072 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:55:38.071+0000 7f408dfb3700 1 -- 172.21.15.137:0/646623707 wait complete. 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.293 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:38.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:38.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:38.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:38.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:38.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:55:38.693 DEBUG:teuthology.orchestra.run.smithi137:osd.4> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T14:55:38.696 INFO:tasks.cephadm:Deploying osd.5 on smithi138 with /dev/vg_nvme/lv_3... 2024-04-25T14:55:38.696 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T14:55:38.938 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:55:39.347 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:39 smithi138 ceph-mon[32875]: from='osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:55:39.348 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:39 smithi138 ceph-mon[32875]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:55:39.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:39 smithi094 ceph-mon[27862]: from='osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:55:39.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:39 smithi094 ceph-mon[27862]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:55:39.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:39 smithi137 ceph-mon[32439]: from='osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:55:39.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:39 smithi137 ceph-mon[32439]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T14:55:40.366 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:40 smithi138 ceph-mon[32875]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:55:40.366 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:40 smithi138 ceph-mon[32875]: from='osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:55:40.366 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:40 smithi138 ceph-mon[32875]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:55:40.367 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:40.367 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:40 smithi138 ceph-mon[32875]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:55:40.367 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:40 smithi138 ceph-mon[32875]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:40.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:40 smithi094 ceph-mon[27862]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:55:40.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:40 smithi094 ceph-mon[27862]: from='osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:55:40.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:40 smithi094 ceph-mon[27862]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:55:40.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:40.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:40 smithi094 ceph-mon[27862]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:55:40.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:40 smithi094 ceph-mon[27862]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:40.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:40 smithi137 ceph-mon[32439]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T14:55:40.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:40 smithi137 ceph-mon[32439]: from='osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:55:40.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:40 smithi137 ceph-mon[32439]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T14:55:40.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:40.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:40 smithi137 ceph-mon[32439]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:55:40.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:40 smithi137 ceph-mon[32439]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T14:55:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T14:55:40.959 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T14:55:40.959 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T14:55:40.959 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124991 s, 839 MB/s 2024-04-25T14:55:40.960 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T14:55:41.257 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:41 smithi138 ceph-mon[32875]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:55:41.257 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:41 smithi138 ceph-mon[32875]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:55:41.257 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:41.257 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:41 smithi138 ceph-mon[32875]: from='osd.4 ' entity='osd.4' 2024-04-25T14:55:41.257 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:41.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:41 smithi137 ceph-mon[32439]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:55:41.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:41 smithi137 ceph-mon[32439]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:55:41.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:41.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:41 smithi137 ceph-mon[32439]: from='osd.4 ' entity='osd.4' 2024-04-25T14:55:41.372 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:41.372 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:55:41 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T14:55:41.108+0000 7f27a672d700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T14:55:41.373 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 14:55:41 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T14:55:41.159+0000 7f279e518700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:55:41.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:41 smithi094 ceph-mon[27862]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:55:41.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:41 smithi094 ceph-mon[27862]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T14:55:41.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:41.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:41 smithi094 ceph-mon[27862]: from='osd.4 ' entity='osd.4' 2024-04-25T14:55:41.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:41.527 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-04-25T14:55:41.761 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:55:42.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:55:42.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:55:42.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] boot 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:55:42.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:42.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:55:42.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:55:42.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:42.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] boot 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:55:42.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:42.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:55:42.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:55:42.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T14:55:42.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: osd.4 [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] boot 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T14:55:42.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T14:55:43.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:43 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi137 2024-04-25T14:55:43.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:43 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.346+0000 7fae54d39700 1 -- 172.21.15.138:0/3563004760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae50103540 msgr2=0x7fae50109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.346+0000 7fae54d39700 1 --2- 172.21.15.138:0/3563004760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae50103540 0x7fae50109df0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fae40009990 tx=0x7fae40009ca0 comp rx=0 tx=0).stop 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 -- 172.21.15.138:0/3563004760 shutdown_connections 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 --2- 172.21.15.138:0/3563004760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae50103540 0x7fae50109df0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 --2- 172.21.15.138:0/3563004760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae501019a0 0x7fae50101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 --2- 172.21.15.138:0/3563004760 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae50102ba0 0x7fae50103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 -- 172.21.15.138:0/3563004760 >> 172.21.15.138:0/3563004760 conn(0x7fae500fcf40 msgr2=0x7fae500ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 -- 172.21.15.138:0/3563004760 shutdown_connections 2024-04-25T14:55:43.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 -- 172.21.15.138:0/3563004760 wait complete. 2024-04-25T14:55:43.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 Processor -- start 2024-04-25T14:55:43.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.347+0000 7fae54d39700 1 -- start start 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae501019a0 0x7fae50198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae50102ba0 0x7fae501991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae50103540 0x7fae501a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fae5010d4d0 con 0x7fae50103540 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fae5010d350 con 0x7fae50102ba0 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae5010d650 con 0x7fae501019a0 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae501019a0 0x7fae50198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae501019a0 0x7fae50198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34772/0 (socket says 172.21.15.138:34772) 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 -- 172.21.15.138:0/1229545817 learned_addr learned my addr 172.21.15.138:0/1229545817 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4dd9b700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae50102ba0 0x7fae501991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 -- 172.21.15.138:0/1229545817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae50102ba0 msgr2=0x7fae501991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:43.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae50102ba0 0x7fae501991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 -- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae50103540 msgr2=0x7fae501a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae50103540 0x7fae501a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 -- 172.21.15.138:0/1229545817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae501a0890 con 0x7fae501019a0 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae4e59c700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae501019a0 0x7fae50198c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b9c0 tx=0x7fae4800bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae477fe700 1 -- 172.21.15.138:0/1229545817 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48019070 con 0x7fae501019a0 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae477fe700 1 -- 172.21.15.138:0/1229545817 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae48014440 con 0x7fae501019a0 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae501a0b80 con 0x7fae501019a0 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae477fe700 1 -- 172.21.15.138:0/1229545817 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae480125a0 con 0x7fae501019a0 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.348+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae501a10c0 con 0x7fae501019a0 2024-04-25T14:55:43.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.349+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae50069730 con 0x7fae501019a0 2024-04-25T14:55:43.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.350+0000 7fae477fe700 1 -- 172.21.15.138:0/1229545817 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fae4801c070 con 0x7fae501019a0 2024-04-25T14:55:43.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.351+0000 7fae477fe700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fae3c0a3c40 0x7fae3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:55:43.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.351+0000 7fae477fe700 1 -- 172.21.15.138:0/1229545817 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fae480ce7e0 con 0x7fae501019a0 2024-04-25T14:55:43.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.351+0000 7fae4dd9b700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fae3c0a3c40 0x7fae3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:55:43.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.352+0000 7fae4dd9b700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fae3c0a3c40 0x7fae3c0a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae38005950 tx=0x7fae380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:55:43.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.356+0000 7fae477fe700 1 -- 172.21.15.138:0/1229545817 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae4809c230 con 0x7fae501019a0 2024-04-25T14:55:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:43 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi137 2024-04-25T14:55:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:43 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-25T14:55:43.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:43 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi137 2024-04-25T14:55:43.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:43 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi137 to 8544M 2024-04-25T14:55:43.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:55:43.561+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fae5006b8e0 con 0x7fae3c0a3c40 2024-04-25T14:55:44.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:44 smithi137 ceph-mon[32439]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:55:44.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:44 smithi137 ceph-mon[32439]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:44.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:44 smithi137 ceph-mon[32439]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:44.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:44.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:44.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:44.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:44 smithi138 ceph-mon[32875]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:55:44.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:44 smithi138 ceph-mon[32875]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:44.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:44 smithi138 ceph-mon[32875]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:44.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:44.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:44.528 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:44 smithi094 ceph-mon[27862]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T14:55:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:44 smithi094 ceph-mon[27862]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:44 smithi094 ceph-mon[27862]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:55:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:55:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:55:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:46 smithi094 ceph-mon[27862]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:46 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/1338954389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]: dispatch 2024-04-25T14:55:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:46 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]: dispatch 2024-04-25T14:55:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:46 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]': finished 2024-04-25T14:55:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:46 smithi094 ceph-mon[27862]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:55:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:55:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:46 smithi137 ceph-mon[32439]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:46 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/1338954389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]: dispatch 2024-04-25T14:55:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:46 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]: dispatch 2024-04-25T14:55:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:46 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]': finished 2024-04-25T14:55:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:46 smithi137 ceph-mon[32439]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:55:46.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:55:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:46 smithi138 ceph-mon[32875]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:46 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/1338954389' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]: dispatch 2024-04-25T14:55:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:46 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]: dispatch 2024-04-25T14:55:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:46 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79a31103-aabc-450e-8932-cd75be992a2d"}]': finished 2024-04-25T14:55:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:46 smithi138 ceph-mon[32875]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T14:55:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:55:47.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:47 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/813028399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:47 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/813028399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:47.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:47 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/813028399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:55:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:48 smithi137 ceph-mon[32439]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:48 smithi138 ceph-mon[32875]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:48 smithi094 ceph-mon[27862]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:50.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:50 smithi138 ceph-mon[32875]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:50 smithi137 ceph-mon[32439]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:50 smithi094 ceph-mon[27862]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:52.546 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:52 smithi138 ceph-mon[32875]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:52.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:52 smithi137 ceph-mon[32439]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:52 smithi094 ceph-mon[27862]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:54.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:54 smithi138 ceph-mon[32875]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:54.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:54 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:55:54.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:54 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:54.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:54 smithi138 ceph-mon[32875]: Deploying daemon osd.5 on smithi138 2024-04-25T14:55:54.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:54 smithi094 ceph-mon[27862]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:54.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:54 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:55:54.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:54 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:54.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:54 smithi094 ceph-mon[27862]: Deploying daemon osd.5 on smithi138 2024-04-25T14:55:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:54 smithi137 ceph-mon[32439]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:54 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T14:55:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:54 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:55:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:54 smithi137 ceph-mon[32439]: Deploying daemon osd.5 on smithi138 2024-04-25T14:55:56.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:56 smithi138 ceph-mon[32875]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:56.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:56 smithi094 ceph-mon[27862]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:56 smithi137 ceph-mon[32439]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:58.600 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:58 smithi138 ceph-mon[32875]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:58.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:58 smithi094 ceph-mon[27862]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:55:58.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:58 smithi137 ceph-mon[32439]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:00.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:59 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:00.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:59 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:00.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:59 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:00.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:55:59 smithi138 ceph-mon[32875]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:00.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:59 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:00.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:59 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:00.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:59 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:00.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:55:59 smithi094 ceph-mon[27862]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:00.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:59 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:00.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:59 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:00.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:59 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:00.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:55:59 smithi137 ceph-mon[32439]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:02.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.296+0000 7fae477fe700 1 -- 172.21.15.138:0/1229545817 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae5006b8e0 con 0x7fae3c0a3c40 2024-04-25T14:56:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 5 on host 'smithi138' 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fae3c0a3c40 msgr2=0x7fae3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fae3c0a3c40 0x7fae3c0a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fae38005950 tx=0x7fae380058e0 comp rx=0 tx=0).stop 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae501019a0 msgr2=0x7fae50198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae501019a0 0x7fae50198c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b9c0 tx=0x7fae4800bd80 comp rx=0 tx=0).stop 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 shutdown_connections 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fae3c0a3c40 0x7fae3c0a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae50103540 0x7fae501a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.298+0000 7fae54d39700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae501019a0 0x7fae50198c90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:02.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.299+0000 7fae54d39700 1 --2- 172.21.15.138:0/1229545817 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fae50102ba0 0x7fae501991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:02.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.299+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 >> 172.21.15.138:0/1229545817 conn(0x7fae500fcf40 msgr2=0x7fae5006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:02.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.299+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 shutdown_connections 2024-04-25T14:56:02.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:56:02.299+0000 7fae54d39700 1 -- 172.21.15.138:0/1229545817 wait complete. 2024-04-25T14:56:02.619 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:02 smithi138 ceph-mon[32875]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:02.619 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.620 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.620 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.620 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:02 smithi137 ceph-mon[32439]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:02 smithi094 ceph-mon[27862]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:02.969 DEBUG:teuthology.orchestra.run.smithi138:osd.5> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T14:56:02.972 INFO:tasks.cephadm:Deploying osd.6 on smithi094 with /dev/vg_nvme/lv_2... 2024-04-25T14:56:02.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:56:03.408 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:56:03.722 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:03 smithi094 ceph-mon[27862]: from='osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:56:03.723 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:03 smithi094 ceph-mon[27862]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:56:03.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:03 smithi137 ceph-mon[32439]: from='osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:56:03.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:03 smithi137 ceph-mon[32439]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:56:03.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:03 smithi138 ceph-mon[32875]: from='osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:56:03.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:03 smithi138 ceph-mon[32875]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T14:56:04.658 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:04 smithi094 ceph-mon[27862]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:04.658 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:04 smithi094 ceph-mon[27862]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:56:04.658 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:04 smithi094 ceph-mon[27862]: from='osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:56:04.658 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:04 smithi094 ceph-mon[27862]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:56:04.658 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:04.658 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:04 smithi094 ceph-mon[27862]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:56:04.950 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-mon[32875]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:04.950 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-mon[32875]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:56:04.950 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-mon[32875]: from='osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:56:04.950 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-mon[32875]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:56:04.950 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:04.950 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-mon[32875]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:56:04.951 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T14:56:04.478+0000 7fa79f56e700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T14:56:04.951 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 14:56:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T14:56:04.487+0000 7fa79b361700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:56:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:04 smithi137 ceph-mon[32439]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:04 smithi137 ceph-mon[32439]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T14:56:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:04 smithi137 ceph-mon[32439]: from='osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:56:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:04 smithi137 ceph-mon[32439]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T14:56:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:04 smithi137 ceph-mon[32439]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:56:05.516 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:05 smithi138 ceph-mon[32875]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:56:05.516 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:05 smithi138 ceph-mon[32875]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:56:05.516 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:05.523 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:05 smithi094 ceph-mon[27862]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:56:05.523 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:05 smithi094 ceph-mon[27862]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:56:05.523 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:05 smithi137 ceph-mon[32439]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:56:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:05 smithi137 ceph-mon[32439]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T14:56:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:56:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:56:06.132 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T14:56:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T14:56:06.132 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0486592 s, 215 MB/s 2024-04-25T14:56:06.132 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T14:56:06.784 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:56:06.784 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:56:06.784 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] boot 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi138 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:06.785 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:06.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.786 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:06 smithi094 ceph-mon[27862]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:56:06.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:56:06.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:56:06.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] boot 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi138 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:06.860 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:06.860 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:06.860 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.860 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:06 smithi138 ceph-mon[32875]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:56:06.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-04-25T14:56:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:56:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:56:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 155 MiB used, 447 GiB / 447 GiB avail 2024-04-25T14:56:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: osd.5 [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] boot 2024-04-25T14:56:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T14:56:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T14:56:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi138 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi138 to 8544M 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:06 smithi137 ceph-mon[32439]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T14:56:07.392 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:56:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:08 smithi094 ceph-mon[27862]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:08 smithi094 ceph-mon[27862]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:56:08.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:08 smithi138 ceph-mon[32875]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:08 smithi138 ceph-mon[32875]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:56:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:08 smithi137 ceph-mon[32439]: pgmap v153: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:08 smithi137 ceph-mon[32439]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 -- 172.21.15.94:0/789806154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc105ee0 msgr2=0x7fa5cc106340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/789806154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc105ee0 0x7fa5cc106340 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009920 tx=0x7fa5b4009c30 comp rx=0 tx=0).stop 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 -- 172.21.15.94:0/789806154 shutdown_connections 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/789806154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc106880 0x7fa5cc10d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/789806154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5cc104ce0 0x7fa5cc105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/789806154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc105ee0 0x7fa5cc106340 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 -- 172.21.15.94:0/789806154 >> 172.21.15.94:0/789806154 conn(0x7fa5cc1002a0 msgr2=0x7fa5cc1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 -- 172.21.15.94:0/789806154 shutdown_connections 2024-04-25T14:56:09.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.562+0000 7fa5d1a00700 1 -- 172.21.15.94:0/789806154 wait complete. 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.563+0000 7fa5d1a00700 1 Processor -- start 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.563+0000 7fa5d1a00700 1 -- start start 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5d1a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc104ce0 0x7fa5cc19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5d1a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc105ee0 0x7fa5cc19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5d1a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5cc106880 0x7fa5cc1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5d1a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc110750 con 0x7fa5cc104ce0 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5d1a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1105d0 con 0x7fa5cc106880 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5d1a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc1108d0 con 0x7fa5cc105ee0 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5caa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc104ce0 0x7fa5cc19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:09.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5caa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc104ce0 0x7fa5cc19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36360/0 (socket says 172.21.15.94:36360) 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5caa00700 1 -- 172.21.15.94:0/1378514525 learned_addr learned my addr 172.21.15.94:0/1378514525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5c2000700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc105ee0 0x7fa5cc19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.564+0000 7fa5cb400700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5cc106880 0x7fa5cc1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c2000700 1 -- 172.21.15.94:0/1378514525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5cc106880 msgr2=0x7fa5cc1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c2000700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5cc106880 0x7fa5cc1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c2000700 1 -- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc104ce0 msgr2=0x7fa5cc19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c2000700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc104ce0 0x7fa5cc19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c2000700 1 -- 172.21.15.94:0/1378514525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc1a2a80 con 0x7fa5cc105ee0 2024-04-25T14:56:09.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5caa00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc104ce0 0x7fa5cc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:56:09.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c2000700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc105ee0 0x7fa5cc19b360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4000c00 tx=0x7fa5b4005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:09.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c3e00700 1 -- 172.21.15.94:0/1378514525 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4027070 con 0x7fa5cc105ee0 2024-04-25T14:56:09.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c3e00700 1 -- 172.21.15.94:0/1378514525 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5b4005510 con 0x7fa5cc105ee0 2024-04-25T14:56:09.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc1a2d10 con 0x7fa5cc105ee0 2024-04-25T14:56:09.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.565+0000 7fa5c3e00700 1 -- 172.21.15.94:0/1378514525 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40049e0 con 0x7fa5cc105ee0 2024-04-25T14:56:09.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.566+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a31c0 con 0x7fa5cc105ee0 2024-04-25T14:56:09.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.567+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc069730 con 0x7fa5cc105ee0 2024-04-25T14:56:09.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.568+0000 7fa5c3e00700 1 -- 172.21.15.94:0/1378514525 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa5b402a070 con 0x7fa5cc105ee0 2024-04-25T14:56:09.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.570+0000 7fa5c3e00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:09.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.570+0000 7fa5c3e00700 1 -- 172.21.15.94:0/1378514525 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7fa5b40cff80 con 0x7fa5cc105ee0 2024-04-25T14:56:09.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.570+0000 7fa5caa00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:09.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.570+0000 7fa5caa00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009e60 tx=0x7fa5bc00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:09.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.574+0000 7fa5c3e00700 1 -- 172.21.15.94:0/1378514525 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5b405eff0 con 0x7fa5cc105ee0 2024-04-25T14:56:09.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:09 smithi094 ceph-mon[27862]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:56:09.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:09.813+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa5cc062e80 con 0x7fa5b00a3cc0 2024-04-25T14:56:09.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:09 smithi138 ceph-mon[32875]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:56:09.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:09 smithi137 ceph-mon[32439]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T14:56:10.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:10 smithi094 ceph-mon[27862]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:10.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:10 smithi094 ceph-mon[27862]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:56:10.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:10 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:10.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:10 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:10.852 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:10 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:10.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:10 smithi138 ceph-mon[32875]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:10 smithi138 ceph-mon[32875]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:56:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:10 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:10 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:10 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:10 smithi137 ceph-mon[32439]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:10 smithi137 ceph-mon[32439]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T14:56:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:10 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:10 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:10 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:11.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:11 smithi094 ceph-mon[27862]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:11.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:11 smithi138 ceph-mon[32875]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:11 smithi137 ceph-mon[32439]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:12.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:12 smithi094 ceph-mon[27862]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:12.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:12 smithi138 ceph-mon[32875]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:12 smithi137 ceph-mon[32439]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:14 smithi138 ceph-mon[32875]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:56:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:14 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4231304943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27a3f85-5246-40dc-bbda-111d7e2f806b"}]: dispatch 2024-04-25T14:56:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:14 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4231304943' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27a3f85-5246-40dc-bbda-111d7e2f806b"}]': finished 2024-04-25T14:56:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:14 smithi138 ceph-mon[32875]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T14:56:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:14 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3049574042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:14 smithi094 ceph-mon[27862]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:56:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:14 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4231304943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27a3f85-5246-40dc-bbda-111d7e2f806b"}]: dispatch 2024-04-25T14:56:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:14 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4231304943' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27a3f85-5246-40dc-bbda-111d7e2f806b"}]': finished 2024-04-25T14:56:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:14 smithi094 ceph-mon[27862]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T14:56:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:14 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3049574042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:14 smithi137 ceph-mon[32439]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:56:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:14 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4231304943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a27a3f85-5246-40dc-bbda-111d7e2f806b"}]: dispatch 2024-04-25T14:56:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:14 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4231304943' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a27a3f85-5246-40dc-bbda-111d7e2f806b"}]': finished 2024-04-25T14:56:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:14 smithi137 ceph-mon[32439]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T14:56:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:14 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3049574042' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:16.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:16 smithi138 ceph-mon[32875]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T14:56:16.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:16 smithi094 ceph-mon[27862]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T14:56:16.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:16 smithi137 ceph-mon[32439]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T14:56:18.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:18 smithi138 ceph-mon[32875]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:56:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:18 smithi094 ceph-mon[27862]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:56:18.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:18 smithi137 ceph-mon[32439]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:56:20.839 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:20 smithi094 ceph-mon[27862]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:56:20.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:20 smithi138 ceph-mon[32875]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:56:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:20 smithi137 ceph-mon[32439]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:56:22.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:22 smithi094 ceph-mon[27862]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:56:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:22 smithi138 ceph-mon[32875]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:56:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:22 smithi137 ceph-mon[32439]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:56:23.855 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:56:23.855 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:23.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:56:23.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:23.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T14:56:23.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:24 smithi094 ceph-mon[27862]: Deploying daemon osd.6 on smithi094 2024-04-25T14:56:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:24 smithi094 ceph-mon[27862]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:24 smithi138 ceph-mon[32875]: Deploying daemon osd.6 on smithi094 2024-04-25T14:56:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:24 smithi138 ceph-mon[32875]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:24 smithi137 ceph-mon[32439]: Deploying daemon osd.6 on smithi094 2024-04-25T14:56:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:24 smithi137 ceph-mon[32439]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:26.923 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:26 smithi094 ceph-mon[27862]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:26.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:26 smithi138 ceph-mon[32875]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:26 smithi137 ceph-mon[32439]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:28.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:28 smithi138 ceph-mon[32875]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:28.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:28 smithi094 ceph-mon[27862]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:28 smithi137 ceph-mon[32439]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:30.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:30 smithi138 ceph-mon[32875]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:30.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:30 smithi094 ceph-mon[27862]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:30 smithi137 ceph-mon[32439]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:31.908 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:31.908 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:31.908 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:31.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:31 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:31.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:31 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:31.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:31 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:31 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:31 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:31 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:32.883 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:32 smithi137 ceph-mon[32439]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:32.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:32 smithi138 ceph-mon[32875]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:32.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:32 smithi094 ceph-mon[27862]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:34.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:34 smithi094 ceph-mon[27862]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:34.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:34.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:34.825 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:34 smithi138 ceph-mon[32875]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:34 smithi137 ceph-mon[32439]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:35.760 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:35 smithi094 ceph-mon[27862]: from='osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:56:35.760 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:35 smithi094 ceph-mon[27862]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:56:35.760 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 6 on host 'smithi094' 2024-04-25T14:56:35.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.759+0000 7fa5c3e00700 1 -- 172.21.15.94:0/1378514525 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa5cc062e80 con 0x7fa5b00a3cc0 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.763+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa5b00a3cc0 msgr2=0x7fa5b00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.763+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009e60 tx=0x7fa5bc00b040 comp rx=0 tx=0).stop 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.763+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc105ee0 msgr2=0x7fa5cc19b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.763+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc105ee0 0x7fa5cc19b360 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4000c00 tx=0x7fa5b4005270 comp rx=0 tx=0).stop 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 shutdown_connections 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa5b00a3cc0 0x7fa5b00a6180 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa5cc106880 0x7fa5cc1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:35.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5cc104ce0 0x7fa5cc19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:35.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 --2- 172.21.15.94:0/1378514525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5cc105ee0 0x7fa5cc19b360 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:35.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 >> 172.21.15.94:0/1378514525 conn(0x7fa5cc1002a0 msgr2=0x7fa5cc1092b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:35.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 shutdown_connections 2024-04-25T14:56:35.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:56:35.764+0000 7fa5d1a00700 1 -- 172.21.15.94:0/1378514525 wait complete. 2024-04-25T14:56:35.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:35 smithi138 ceph-mon[32875]: from='osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:56:35.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:35 smithi138 ceph-mon[32875]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:56:35.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:35 smithi137 ceph-mon[32439]: from='osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:56:35.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:35 smithi137 ceph-mon[32439]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T14:56:36.464 DEBUG:teuthology.orchestra.run.smithi094:osd.6> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T14:56:36.469 INFO:tasks.cephadm:Deploying osd.7 on smithi137 with /dev/vg_nvme/lv_2... 2024-04-25T14:56:36.469 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:56:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:56:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: from='osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:56:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:56:36.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:36.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:56:36.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:36.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:36.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:36 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:36.715 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:56:36.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:36.827 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:56:36.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: from='osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:56:36.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:56:36.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:36.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:56:36.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:36.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:36.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:36 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:36.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:36.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T14:56:36.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: from='osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:56:36.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T14:56:36.858 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:36.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:56:36.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:36.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:36.859 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:36 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:37 smithi094 ceph-mon[27862]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:56:37.599 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:37 smithi094 ceph-mon[27862]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:56:37.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:37.600 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:37 smithi138 ceph-mon[32875]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:37 smithi138 ceph-mon[32875]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:37 smithi137 ceph-mon[32439]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:37 smithi137 ceph-mon[32439]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:37.957 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:37.962 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:56:37 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T14:56:37.661+0000 7f1dcea00700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T14:56:37.962 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 14:56:37 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T14:56:37.687+0000 7f1dc4800700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:56:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:56:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:56:38.675 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T14:56:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T14:56:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01278 s, 820 MB/s 2024-04-25T14:56:38.675 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T14:56:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:38 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:56:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:38 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:56:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:38 smithi138 ceph-mon[32875]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:38 smithi138 ceph-mon[32875]: from='osd.6 ' entity='osd.6' 2024-04-25T14:56:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:38 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:56:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:38 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:56:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:38 smithi094 ceph-mon[27862]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:38 smithi094 ceph-mon[27862]: from='osd.6 ' entity='osd.6' 2024-04-25T14:56:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:38 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:56:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:38 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:56:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:38 smithi137 ceph-mon[32439]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:38 smithi137 ceph-mon[32439]: from='osd.6 ' entity='osd.6' 2024-04-25T14:56:39.267 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_2 2024-04-25T14:56:39.501 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:56:39.868 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:39 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:39.868 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:39 smithi137 ceph-mon[32439]: osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] boot 2024-04-25T14:56:39.868 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:39 smithi137 ceph-mon[32439]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T14:56:39.868 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:39 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:39.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:39 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:39.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:39 smithi138 ceph-mon[32875]: osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] boot 2024-04-25T14:56:39.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:39 smithi138 ceph-mon[32875]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T14:56:39.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:39 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:39 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:39 smithi094 ceph-mon[27862]: osd.6 [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] boot 2024-04-25T14:56:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:39 smithi094 ceph-mon[27862]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T14:56:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:39 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T14:56:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:40 smithi094 ceph-mon[27862]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:40 smithi094 ceph-mon[27862]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:56:40.904 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:40 smithi137 ceph-mon[32439]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:40.905 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:40 smithi137 ceph-mon[32439]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:56:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:40 smithi138 ceph-mon[32875]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-25T14:56:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:40 smithi138 ceph-mon[32875]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T14:56:41.134 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 -- 172.21.15.137:0/478541404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102f80 msgr2=0x7f698c1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 --2- 172.21.15.137:0/478541404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102f80 0x7f698c1033a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f697c009990 tx=0x7f697c009ca0 comp rx=0 tx=0).stop 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 -- 172.21.15.137:0/478541404 shutdown_connections 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 --2- 172.21.15.137:0/478541404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698c102660 0x7f698c109f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 --2- 172.21.15.137:0/478541404 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102f80 0x7f698c1033a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 --2- 172.21.15.137:0/478541404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c101cc0 0x7f698c102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 -- 172.21.15.137:0/478541404 >> 172.21.15.137:0/478541404 conn(0x7f698c0fc710 msgr2=0x7f698c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 -- 172.21.15.137:0/478541404 shutdown_connections 2024-04-25T14:56:41.135 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 -- 172.21.15.137:0/478541404 wait complete. 2024-04-25T14:56:41.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 Processor -- start 2024-04-25T14:56:41.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.127+0000 7f6991a0a700 1 -- start start 2024-04-25T14:56:41.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c101cc0 0x7f698c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:41.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102660 0x7f698c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:41.136 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698c102f80 0x7f698c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10d480 con 0x7f698c102f80 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10d300 con 0x7f698c102660 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f698c10d600 con 0x7f698c101cc0 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102660 0x7f698c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102660 0x7f698c199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:35710/0 (socket says 172.21.15.137:35710) 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 -- 172.21.15.137:0/519418146 learned_addr learned my addr 172.21.15.137:0/519418146 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698affd700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c101cc0 0x7f698c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:41.137 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 -- 172.21.15.137:0/519418146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c101cc0 msgr2=0x7f698c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c101cc0 0x7f698c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 -- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698c102f80 msgr2=0x7f698c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698b7fe700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698c102f80 0x7f698c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698c102f80 0x7f698c1a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 -- 172.21.15.137:0/519418146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c1a0840 con 0x7f698c102660 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f698a7fc700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102660 0x7f698c199120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f698000e9c0 tx=0x7f698000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6990a08700 1 -- 172.21.15.137:0/519418146 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698000cbd0 con 0x7f698c102660 2024-04-25T14:56:41.138 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6990a08700 1 -- 172.21.15.137:0/519418146 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698000cd70 con 0x7f698c102660 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c1a0b30 con 0x7f698c102660 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6990a08700 1 -- 172.21.15.137:0/519418146 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980010430 con 0x7f698c102660 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.128+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c1a1070 con 0x7f698c102660 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.129+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c069730 con 0x7f698c102660 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.130+0000 7f6990a08700 1 -- 172.21.15.137:0/519418146 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6980004750 con 0x7f698c102660 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.131+0000 7f6990a08700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f69740a3cc0 0x7f69740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.131+0000 7f6990a08700 1 -- 172.21.15.137:0/519418146 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f69800cfc40 con 0x7f698c102660 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.131+0000 7f698affd700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f69740a3cc0 0x7f69740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:56:41.139 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.132+0000 7f698affd700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f69740a3cc0 0x7f69740a6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b850 tx=0x7f697c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:56:41.140 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.135+0000 7f6990a08700 1 -- 172.21.15.137:0/519418146 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6980053ff0 con 0x7f698c102660 2024-04-25T14:56:41.344 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:56:41.343+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f698c0fe8e0 con 0x7f69740a3cc0 2024-04-25T14:56:41.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:41.951 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:41.951 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:41.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:41.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:41.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:41.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:56:41.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:56:41.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.923 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:42.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:42.924 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:56:42.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:42.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:56:42.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:42.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:42.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:56:42.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:43.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:43 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi094 2024-04-25T14:56:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:43 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T14:56:43.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:43 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi094 2024-04-25T14:56:43.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:43 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T14:56:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:43 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi094 2024-04-25T14:56:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:43 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi094 to 5696M 2024-04-25T14:56:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:44 smithi137 ceph-mon[32439]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:44 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/3583345793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]: dispatch 2024-04-25T14:56:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:44 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]: dispatch 2024-04-25T14:56:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:44 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]': finished 2024-04-25T14:56:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:44 smithi137 ceph-mon[32439]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T14:56:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:44 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:56:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:44 smithi138 ceph-mon[32875]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:44 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/3583345793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]: dispatch 2024-04-25T14:56:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:44 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]: dispatch 2024-04-25T14:56:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:44 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]': finished 2024-04-25T14:56:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:44 smithi138 ceph-mon[32875]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T14:56:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:44 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:56:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:44 smithi094 ceph-mon[27862]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:44 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/3583345793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]: dispatch 2024-04-25T14:56:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:44 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]: dispatch 2024-04-25T14:56:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:44 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd"}]': finished 2024-04-25T14:56:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:44 smithi094 ceph-mon[27862]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T14:56:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:44 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:56:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:45 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/1409395468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:45.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:45 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/1409395468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:45.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:45 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/1409395468' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:56:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:46 smithi138 ceph-mon[32875]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:46.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:46 smithi094 ceph-mon[27862]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:46 smithi137 ceph-mon[32439]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:48 smithi138 ceph-mon[32875]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:48 smithi094 ceph-mon[27862]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:48 smithi137 ceph-mon[32439]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:49.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:49 smithi138 ceph-mon[32875]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:49.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:49 smithi094 ceph-mon[27862]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:49.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:49 smithi137 ceph-mon[32439]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:52.565 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:52 smithi137 ceph-mon[32439]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:52.565 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:56:52.565 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:52.565 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:52 smithi137 ceph-mon[32439]: Deploying daemon osd.7 on smithi137 2024-04-25T14:56:52.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:52 smithi138 ceph-mon[32875]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:56:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:52 smithi138 ceph-mon[32875]: Deploying daemon osd.7 on smithi137 2024-04-25T14:56:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:52 smithi094 ceph-mon[27862]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T14:56:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:56:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:52 smithi094 ceph-mon[27862]: Deploying daemon osd.7 on smithi137 2024-04-25T14:56:54.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:54 smithi137 ceph-mon[32439]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:54.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:54 smithi138 ceph-mon[32875]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:54.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:54 smithi094 ceph-mon[27862]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:56.541 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:56 smithi137 ceph-mon[32439]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:56.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:56 smithi138 ceph-mon[32875]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:56.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:56 smithi094 ceph-mon[27862]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:56:58 smithi138 ceph-mon[32875]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:58.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:58.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:58.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:58.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:56:58 smithi094 ceph-mon[27862]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:56:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:56:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:56:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:56:58 smithi137 ceph-mon[32439]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:00.701 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:00 smithi137 ceph-mon[32439]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:00.701 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.701 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.701 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:00.701 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:00.701 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:00 smithi138 ceph-mon[32875]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:00 smithi094 ceph-mon[27862]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:01.031 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 7 on host 'smithi137' 2024-04-25T14:57:01.032 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.031+0000 7f6990a08700 1 -- 172.21.15.137:0/519418146 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f698c0fe8e0 con 0x7f69740a3cc0 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f69740a3cc0 msgr2=0x7f69740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f69740a3cc0 0x7f69740a6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b850 tx=0x7f697c01a040 comp rx=0 tx=0).stop 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102660 msgr2=0x7f698c199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102660 0x7f698c199120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f698000e9c0 tx=0x7f698000ed80 comp rx=0 tx=0).stop 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 shutdown_connections 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f69740a3cc0 0x7f69740a6180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f698c102f80 0x7f698c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:01.034 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f698c101cc0 0x7f698c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:01.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 --2- 172.21.15.137:0/519418146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f698c102660 0x7f698c199120 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:01.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 >> 172.21.15.137:0/519418146 conn(0x7f698c0fc710 msgr2=0x7f698c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:01.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 shutdown_connections 2024-04-25T14:57:01.035 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:57:01.033+0000 7f6991a0a700 1 -- 172.21.15.137:0/519418146 wait complete. 2024-04-25T14:57:01.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:01 smithi137 ceph-mon[32439]: from='osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:57:01.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:01 smithi137 ceph-mon[32439]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:57:01.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:01.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:01.671 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:01.701 DEBUG:teuthology.orchestra.run.smithi137:osd.7> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T14:57:01.703 INFO:tasks.cephadm:Deploying osd.8 on smithi138 with /dev/vg_nvme/lv_2... 2024-04-25T14:57:01.703 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T14:57:01.733 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:01 smithi138 ceph-mon[32875]: from='osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:57:01.733 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:01 smithi138 ceph-mon[32875]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:57:01.733 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:01.733 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:01.733 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:01.950 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:57:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:01 smithi094 ceph-mon[27862]: from='osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:57:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:01 smithi094 ceph-mon[27862]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T14:57:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:01.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:01.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:02 smithi138 ceph-mon[32875]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:02 smithi138 ceph-mon[32875]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:57:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:02 smithi138 ceph-mon[32875]: from='osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:57:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:02 smithi138 ceph-mon[32875]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:57:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:02 smithi138 ceph-mon[32875]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:57:02.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-mon[32439]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:02.811 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-mon[32439]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:57:02.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-mon[32439]: from='osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:57:02.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-mon[32439]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:57:02.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:02.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-mon[32439]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:57:02.812 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T14:57:02.501+0000 7faea3eca700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T14:57:02.812 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 14:57:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T14:57:02.513+0000 7fae9fcbd700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:57:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:02 smithi094 ceph-mon[27862]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:02 smithi094 ceph-mon[27862]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T14:57:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:02 smithi094 ceph-mon[27862]: from='osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:57:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:02 smithi094 ceph-mon[27862]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T14:57:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:02 smithi094 ceph-mon[27862]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:57:03.750 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:03 smithi138 ceph-mon[32875]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:57:03.750 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:03 smithi138 ceph-mon[32875]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:57:03.750 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:03 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:03.750 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T14:57:03.750 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T14:57:03.750 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T14:57:03.751 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T14:57:03.751 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122305 s, 857 MB/s 2024-04-25T14:57:03.751 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T14:57:03.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:03 smithi137 ceph-mon[32439]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:57:03.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:03 smithi137 ceph-mon[32439]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:57:03.908 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:03 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:03.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:03 smithi094 ceph-mon[27862]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:57:03.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:03 smithi094 ceph-mon[27862]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T14:57:03.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:03 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:04.317 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-04-25T14:57:04.605 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:57:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:04 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:57:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:04 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:57:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:04 smithi138 ceph-mon[32875]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:04 smithi138 ceph-mon[32875]: osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] boot 2024-04-25T14:57:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:04 smithi138 ceph-mon[32875]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T14:57:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:04.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:04 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:57:04.812 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:04 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:57:04.813 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:04 smithi137 ceph-mon[32439]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:04.813 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:04 smithi137 ceph-mon[32439]: osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] boot 2024-04-25T14:57:04.813 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:04 smithi137 ceph-mon[32439]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T14:57:04.813 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:04 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:57:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:04 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:57:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:04 smithi094 ceph-mon[27862]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-04-25T14:57:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:04 smithi094 ceph-mon[27862]: osd.7 [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] boot 2024-04-25T14:57:04.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:04 smithi094 ceph-mon[27862]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T14:57:04.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi137 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.557 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-25T14:57:05.558 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.558 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:05.558 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:05.558 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:05 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:57:05.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi137 2024-04-25T14:57:05.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:05.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:05 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T14:57:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi137 2024-04-25T14:57:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:05.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi137 to 5696M 2024-04-25T14:57:05.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:05.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:05.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:05.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:05 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:06.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 -- 172.21.15.138:0/3461394912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8103540 msgr2=0x7f1ae8105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:06.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 --2- 172.21.15.138:0/3461394912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8103540 0x7f1ae8105930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009990 tx=0x7f1ad8009ca0 comp rx=0 tx=0).stop 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 -- 172.21.15.138:0/3461394912 shutdown_connections 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 --2- 172.21.15.138:0/3461394912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae81087a0 0x7f1ae810ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 --2- 172.21.15.138:0/3461394912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8103540 0x7f1ae8105930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 --2- 172.21.15.138:0/3461394912 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae8105e70 0x7f1ae8108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 -- 172.21.15.138:0/3461394912 >> 172.21.15.138:0/3461394912 conn(0x7f1ae80fcf70 msgr2=0x7f1ae80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 -- 172.21.15.138:0/3461394912 shutdown_connections 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 -- 172.21.15.138:0/3461394912 wait complete. 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.136+0000 7f1aedc35700 1 Processor -- start 2024-04-25T14:57:06.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1aedc35700 1 -- start start 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1aedc35700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae8103540 0x7f1ae8198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1aedc35700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8105e70 0x7f1ae8199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1aedc35700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae81087a0 0x7f1ae81a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1aedc35700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae810e5d0 con 0x7f1ae8103540 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1aedc35700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae810e450 con 0x7f1ae81087a0 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1aedc35700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae810e750 con 0x7f1ae8105e70 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8105e70 0x7f1ae8199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8105e70 0x7f1ae8199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38268/0 (socket says 172.21.15.138:38268) 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 -- 172.21.15.138:0/2706853482 learned_addr learned my addr 172.21.15.138:0/2706853482 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae7fff700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae81087a0 0x7f1ae81a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 -- 172.21.15.138:0/2706853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae81087a0 msgr2=0x7f1ae81a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae81087a0 0x7f1ae81a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 -- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae8103540 msgr2=0x7f1ae8198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae8103540 0x7f1ae8198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 -- 172.21.15.138:0/2706853482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae81a08f0 con 0x7f1ae8105e70 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae6ffd700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8105e70 0x7f1ae8199200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e9c0 tx=0x7f1adc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.137+0000 7f1ae4ff9700 1 -- 172.21.15.138:0/2706853482 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc00cbd0 con 0x7f1ae8105e70 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.138+0000 7f1ae4ff9700 1 -- 172.21.15.138:0/2706853482 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1adc00cd70 con 0x7f1ae8105e70 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.138+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae81a0be0 con 0x7f1ae8105e70 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.138+0000 7f1ae4ff9700 1 -- 172.21.15.138:0/2706853482 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1adc005610 con 0x7f1ae8105e70 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.138+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae81a10d0 con 0x7f1ae8105e70 2024-04-25T14:57:06.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.138+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae8069730 con 0x7f1ae8105e70 2024-04-25T14:57:06.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.140+0000 7f1ae4ff9700 1 -- 172.21.15.138:0/2706853482 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1adc004750 con 0x7f1ae8105e70 2024-04-25T14:57:06.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.140+0000 7f1ae4ff9700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1ad00a3c40 0x7f1ad00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:06.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.140+0000 7f1ae4ff9700 1 -- 172.21.15.138:0/2706853482 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f1adc0cfdb0 con 0x7f1ae8105e70 2024-04-25T14:57:06.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.141+0000 7f1ae77fe700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1ad00a3c40 0x7f1ad00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:06.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.142+0000 7f1ae77fe700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1ad00a3c40 0x7f1ad00a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80098e0 tx=0x7f1ad800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:06.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.144+0000 7f1ae4ff9700 1 -- 172.21.15.138:0/2706853482 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1adc053ff0 con 0x7f1ae8105e70 2024-04-25T14:57:06.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:06.360+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1ae8062e80 con 0x7f1ad00a3c40 2024-04-25T14:57:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:06 smithi138 ceph-mon[32875]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:06 smithi138 ceph-mon[32875]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:57:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:06.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:06 smithi094 ceph-mon[27862]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:06.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:06 smithi094 ceph-mon[27862]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:57:06.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:06.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:06.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:06 smithi137 ceph-mon[32439]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:06 smithi137 ceph-mon[32439]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T14:57:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:06.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:07.889 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:07 smithi138 ceph-mon[32875]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:07.889 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:07 smithi138 ceph-mon[32875]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:57:07.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:07 smithi094 ceph-mon[27862]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:07.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:07 smithi094 ceph-mon[27862]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:57:07.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:07 smithi137 ceph-mon[32439]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:07.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:07 smithi137 ceph-mon[32439]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T14:57:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:08 smithi138 ceph-mon[32875]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:08 smithi138 ceph-mon[32875]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T14:57:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:08 smithi138 ceph-mon[32875]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:57:08.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:08 smithi094 ceph-mon[27862]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:08.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:08 smithi094 ceph-mon[27862]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T14:57:08.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:08 smithi094 ceph-mon[27862]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:57:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:08 smithi137 ceph-mon[32439]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:08 smithi137 ceph-mon[32439]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T14:57:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:08 smithi137 ceph-mon[32439]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T14:57:09.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:09 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/503934357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]: dispatch 2024-04-25T14:57:09.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:09 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]: dispatch 2024-04-25T14:57:09.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:09 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]': finished 2024-04-25T14:57:09.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:09 smithi138 ceph-mon[32875]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T14:57:09.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:09.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:09 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/3108723949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:09 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/503934357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]: dispatch 2024-04-25T14:57:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:09 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]: dispatch 2024-04-25T14:57:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:09 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]': finished 2024-04-25T14:57:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:09 smithi094 ceph-mon[27862]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T14:57:09.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:09.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:09 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/3108723949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:09.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:09 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/503934357' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]: dispatch 2024-04-25T14:57:09.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:09 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]: dispatch 2024-04-25T14:57:09.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:09 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "571dbd3c-a8f3-425f-a34c-f0f2c168907a"}]': finished 2024-04-25T14:57:09.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:09 smithi137 ceph-mon[32439]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T14:57:09.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:09 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:09.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:09 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/3108723949' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:10 smithi138 ceph-mon[32875]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:10 smithi138 ceph-mon[32875]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-25T14:57:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:10 smithi138 ceph-mon[32875]: Cluster is now healthy 2024-04-25T14:57:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:10 smithi094 ceph-mon[27862]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:10 smithi094 ceph-mon[27862]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-25T14:57:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:10 smithi094 ceph-mon[27862]: Cluster is now healthy 2024-04-25T14:57:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:10 smithi137 ceph-mon[32439]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:10 smithi137 ceph-mon[32439]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-25T14:57:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:10 smithi137 ceph-mon[32439]: Cluster is now healthy 2024-04-25T14:57:11.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:11 smithi138 ceph-mon[32875]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:11.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:11 smithi094 ceph-mon[27862]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:11 smithi137 ceph-mon[32439]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:14.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:14 smithi138 ceph-mon[32875]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:14 smithi094 ceph-mon[27862]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:14 smithi137 ceph-mon[32439]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:16 smithi138 ceph-mon[32875]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:57:16.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:16 smithi094 ceph-mon[27862]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:57:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:16.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:16 smithi137 ceph-mon[32439]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:16.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T14:57:16.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:17.916 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:17 smithi138 ceph-mon[32875]: Deploying daemon osd.8 on smithi138 2024-04-25T14:57:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:17 smithi094 ceph-mon[27862]: Deploying daemon osd.8 on smithi138 2024-04-25T14:57:17.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:17 smithi137 ceph-mon[32439]: Deploying daemon osd.8 on smithi138 2024-04-25T14:57:18.955 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:18 smithi138 ceph-mon[32875]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:18 smithi094 ceph-mon[27862]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:18.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:18 smithi137 ceph-mon[32439]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:20 smithi138 ceph-mon[32875]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:20 smithi094 ceph-mon[27862]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:20 smithi137 ceph-mon[32439]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:22 smithi138 ceph-mon[32875]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:22 smithi094 ceph-mon[27862]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:22 smithi137 ceph-mon[32439]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:24.899 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:24 smithi138 ceph-mon[32875]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:24 smithi094 ceph-mon[27862]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:24 smithi137 ceph-mon[32439]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:24.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.993+0000 7f1ae4ff9700 1 -- 172.21.15.138:0/2706853482 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ae8062e80 con 0x7f1ad00a3c40 2024-04-25T14:57:24.995 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 8 on host 'smithi138' 2024-04-25T14:57:24.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1ad00a3c40 msgr2=0x7f1ad00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:24.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1ad00a3c40 0x7f1ad00a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80098e0 tx=0x7f1ad800b610 comp rx=0 tx=0).stop 2024-04-25T14:57:24.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8105e70 msgr2=0x7f1ae8199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8105e70 0x7f1ae8199200 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00e9c0 tx=0x7f1adc00ecd0 comp rx=0 tx=0).stop 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 shutdown_connections 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1ad00a3c40 0x7f1ad00a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1ae81087a0 0x7f1ae81a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ae8103540 0x7f1ae8198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 --2- 172.21.15.138:0/2706853482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ae8105e70 0x7f1ae8199200 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 >> 172.21.15.138:0/2706853482 conn(0x7f1ae80fcf70 msgr2=0x7f1ae8103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 shutdown_connections 2024-04-25T14:57:24.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:57:24.997+0000 7f1aedc35700 1 -- 172.21.15.138:0/2706853482 wait complete. 2024-04-25T14:57:25.597 DEBUG:teuthology.orchestra.run.smithi138:osd.8> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T14:57:25.599 INFO:tasks.cephadm:Deploying osd.9 on smithi094 with /dev/vg_nvme/lv_1... 2024-04-25T14:57:25.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:57:25.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:25 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:25.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:25 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:25.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:25 smithi094 ceph-mon[27862]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:25.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:25 smithi094 ceph-mon[27862]: from='osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:57:25.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:25 smithi094 ceph-mon[27862]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:57:25.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:25 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:25.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:25 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.050 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:57:26.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:25 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:25 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:25 smithi138 ceph-mon[32875]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:26.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:25 smithi138 ceph-mon[32875]: from='osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:57:26.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:25 smithi138 ceph-mon[32875]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:57:26.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:25 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:25 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:25 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:25 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:25 smithi137 ceph-mon[32439]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:26.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:25 smithi137 ceph-mon[32439]: from='osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:57:26.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:25 smithi137 ceph-mon[32439]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T14:57:26.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:25 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:26.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:25 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:27.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:26 smithi138 ceph-mon[32875]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:57:27.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:26 smithi138 ceph-mon[32875]: from='osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:57:27.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:26 smithi138 ceph-mon[32875]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:57:27.135 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:27.136 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:26 smithi138 ceph-mon[32875]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:57:27.136 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:57:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T14:57:26.961+0000 7f2ebf1c8700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T14:57:27.136 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 14:57:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T14:57:26.972+0000 7f2ebafbb700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:57:27.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:26 smithi094 ceph-mon[27862]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:57:27.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:26 smithi094 ceph-mon[27862]: from='osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:57:27.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:26 smithi094 ceph-mon[27862]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:57:27.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:27.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:26 smithi094 ceph-mon[27862]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:57:27.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:26 smithi137 ceph-mon[32439]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T14:57:27.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:26 smithi137 ceph-mon[32439]: from='osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:57:27.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:26 smithi137 ceph-mon[32439]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T14:57:27.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:27.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:26 smithi137 ceph-mon[32439]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:57:28.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:27 smithi094 ceph-mon[27862]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:57:28.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:27 smithi094 ceph-mon[27862]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:57:28.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:27 smithi094 ceph-mon[27862]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:27 smithi137 ceph-mon[32439]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:57:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:27 smithi137 ceph-mon[32439]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:57:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:27 smithi137 ceph-mon[32439]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:28.298 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:27 smithi138 ceph-mon[32875]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:57:28.298 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:27 smithi138 ceph-mon[32875]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T14:57:28.298 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:28.298 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:27 smithi138 ceph-mon[32875]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-25T14:57:28.749 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:57:28.749 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:57:28.750 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-04-25T14:57:28.750 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-04-25T14:57:28.750 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0400229 s, 262 MB/s 2024-04-25T14:57:28.750 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] boot 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.155 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:29.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:29.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.156 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:28 smithi094 ceph-mon[27862]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T14:57:29.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:57:29.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:57:29.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] boot 2024-04-25T14:57:29.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T14:57:29.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:29.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:29.234 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.234 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:28 smithi137 ceph-mon[32439]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: osd.8 [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] boot 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:57:29.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:29.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:29.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:29.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:28 smithi138 ceph-mon[32875]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T14:57:29.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-04-25T14:57:30.077 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:57:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi138 2024-04-25T14:57:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-25T14:57:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:30.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:30.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:29 smithi137 ceph-mon[32439]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:57:30.241 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi138 2024-04-25T14:57:30.241 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-25T14:57:30.241 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:30.241 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:30.241 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:30.242 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:30.242 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:30.242 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:29 smithi094 ceph-mon[27862]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi138 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi138 to 5696M 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: pgmap v215: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:29 smithi138 ceph-mon[32875]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T14:57:32.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.157+0000 7f2609600700 1 -- 172.21.15.94:0/2390582713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604108790 msgr2=0x7f260410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.157+0000 7f2609600700 1 --2- 172.21.15.94:0/2390582713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604108790 0x7f260410ac00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0098f0 tx=0x7f25fc009c00 comp rx=0 tx=0).stop 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.157+0000 7f2609600700 1 -- 172.21.15.94:0/2390582713 shutdown_connections 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.158+0000 7f2609600700 1 --2- 172.21.15.94:0/2390582713 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604108790 0x7f260410ac00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.158+0000 7f2609600700 1 --2- 172.21.15.94:0/2390582713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 0x7f2604105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.158+0000 7f2609600700 1 --2- 172.21.15.94:0/2390582713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604105e60 0x7f2604108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.158+0000 7f2609600700 1 -- 172.21.15.94:0/2390582713 >> 172.21.15.94:0/2390582713 conn(0x7f26040fcf40 msgr2=0x7f26040ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.158+0000 7f2609600700 1 -- 172.21.15.94:0/2390582713 shutdown_connections 2024-04-25T14:57:32.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.158+0000 7f2609600700 1 -- 172.21.15.94:0/2390582713 wait complete. 2024-04-25T14:57:32.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 Processor -- start 2024-04-25T14:57:32.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 -- start start 2024-04-25T14:57:32.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 0x7f260419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:32.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604105e60 0x7f260419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108790 0x7f26041a2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f260410e550 con 0x7f2604108790 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f260410e3d0 con 0x7f2604105e60 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.159+0000 7f2609600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f260410e6d0 con 0x7f2604103530 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 0x7f260419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2603400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108790 0x7f26041a2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 0x7f260419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:57272/0 (socket says 172.21.15.94:57272) 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 -- 172.21.15.94:0/1758338210 learned_addr learned my addr 172.21.15.94:0/1758338210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f25fa000700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604105e60 0x7f260419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 -- 172.21.15.94:0/1758338210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604105e60 msgr2=0x7f260419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604105e60 0x7f260419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:32.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 -- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108790 msgr2=0x7f26041a2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108790 0x7f26041a2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 -- 172.21.15.94:0/1758338210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26041a2ab0 con 0x7f2604103530 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2603400700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108790 0x7f26041a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.160+0000 7f2602a00700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 0x7f260419ae50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e3f0 tx=0x7f25f400e700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.161+0000 7f25fbe00700 1 -- 172.21.15.94:0/1758338210 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f400ee70 con 0x7f2604103530 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.161+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041a2d40 con 0x7f2604103530 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.161+0000 7f25fbe00700 1 -- 172.21.15.94:0/1758338210 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f4003a00 con 0x7f2604103530 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.161+0000 7f25fbe00700 1 -- 172.21.15.94:0/1758338210 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4004660 con 0x7f2604103530 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.161+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a3210 con 0x7f2604103530 2024-04-25T14:57:32.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.162+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2604069730 con 0x7f2604103530 2024-04-25T14:57:32.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.164+0000 7f25fbe00700 1 -- 172.21.15.94:0/1758338210 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f25f401b030 con 0x7f2604103530 2024-04-25T14:57:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.166+0000 7f25fbe00700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25e80a3c40 0x7f25e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:57:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.166+0000 7f25fbe00700 1 -- 172.21.15.94:0/1758338210 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7f25f40cf530 con 0x7f2604103530 2024-04-25T14:57:32.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.166+0000 7f25fa000700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25e80a3c40 0x7f25e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:57:32.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.167+0000 7f25fa000700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25e80a3c40 0x7f25e80a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25ec009c60 tx=0x7f25ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:57:32.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.170+0000 7f25fbe00700 1 -- 172.21.15.94:0/1758338210 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f25f4022020 con 0x7f2604103530 2024-04-25T14:57:32.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:32.403+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2604062e80 con 0x7f25e80a3c40 2024-04-25T14:57:32.405 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:32 smithi094 ceph-mon[27862]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:57:32.406 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:32 smithi094 ceph-mon[27862]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:32.456 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:32 smithi138 ceph-mon[32875]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:57:32.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:32 smithi138 ceph-mon[32875]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:32.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:32 smithi137 ceph-mon[32439]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T14:57:32.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:32 smithi137 ceph-mon[32439]: pgmap v218: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:33.308 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:33 smithi137 ceph-mon[32439]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:57:33.308 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:33 smithi137 ceph-mon[32439]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:33.308 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:33 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:33.308 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:33 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:33.308 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:33 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:33.327 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:33 smithi094 ceph-mon[27862]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:57:33.327 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:33 smithi094 ceph-mon[27862]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:33.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:33 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:33.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:33 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:33.328 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:33 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:33.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:33 smithi138 ceph-mon[32875]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T14:57:33.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:33 smithi138 ceph-mon[32875]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:57:33.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:33 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:57:33.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:33 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:57:33.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:33 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:34.343 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:34 smithi094 ceph-mon[27862]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:34.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:34 smithi138 ceph-mon[32875]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:34.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:34 smithi137 ceph-mon[32439]: pgmap v220: 1 pgs: 1 remapped+peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:36.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:36 smithi094 ceph-mon[27862]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:57:36.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:36 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/153009150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0b7d2e9-2dd4-4dbf-befd-e860062b42ae"}]: dispatch 2024-04-25T14:57:36.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:36 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/153009150' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0b7d2e9-2dd4-4dbf-befd-e860062b42ae"}]': finished 2024-04-25T14:57:36.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:36 smithi094 ceph-mon[27862]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T14:57:36.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:36 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:57:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:36 smithi138 ceph-mon[32875]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:57:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:36 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/153009150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0b7d2e9-2dd4-4dbf-befd-e860062b42ae"}]: dispatch 2024-04-25T14:57:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:36 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/153009150' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0b7d2e9-2dd4-4dbf-befd-e860062b42ae"}]': finished 2024-04-25T14:57:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:36 smithi138 ceph-mon[32875]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T14:57:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:36 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:57:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:36 smithi137 ceph-mon[32439]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T14:57:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:36 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/153009150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f0b7d2e9-2dd4-4dbf-befd-e860062b42ae"}]: dispatch 2024-04-25T14:57:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:36 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/153009150' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f0b7d2e9-2dd4-4dbf-befd-e860062b42ae"}]': finished 2024-04-25T14:57:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:36 smithi137 ceph-mon[32439]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T14:57:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:36 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:57:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:37 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1387401637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:37.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:37 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1387401637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:37 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1387401637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:57:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:38 smithi138 ceph-mon[32875]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T14:57:38.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:38 smithi094 ceph-mon[27862]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T14:57:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:38 smithi137 ceph-mon[32439]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T14:57:40.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:40 smithi094 ceph-mon[27862]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:57:40.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:40 smithi138 ceph-mon[32875]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:57:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:40 smithi137 ceph-mon[32439]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T14:57:42.602 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:42 smithi094 ceph-mon[27862]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T14:57:42.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:42 smithi138 ceph-mon[32875]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T14:57:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:42 smithi137 ceph-mon[32439]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T14:57:44.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:44 smithi137 ceph-mon[32439]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:57:44.942 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:44 smithi094 ceph-mon[27862]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:57:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:44 smithi138 ceph-mon[32875]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T14:57:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:57:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:57:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:45.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T14:57:45.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:57:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:46 smithi094 ceph-mon[27862]: Deploying daemon osd.9 on smithi094 2024-04-25T14:57:46.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:46 smithi094 ceph-mon[27862]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:46 smithi138 ceph-mon[32875]: Deploying daemon osd.9 on smithi094 2024-04-25T14:57:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:46 smithi138 ceph-mon[32875]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:46 smithi137 ceph-mon[32439]: Deploying daemon osd.9 on smithi094 2024-04-25T14:57:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:46 smithi137 ceph-mon[32439]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:48.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:48 smithi094 ceph-mon[27862]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:48 smithi138 ceph-mon[32875]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:48 smithi137 ceph-mon[32439]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:50.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:50 smithi094 ceph-mon[27862]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:50.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:50 smithi138 ceph-mon[32875]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:50 smithi137 ceph-mon[32439]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:52 smithi094 ceph-mon[27862]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:52 smithi138 ceph-mon[32875]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:52 smithi137 ceph-mon[32439]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:54.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:54 smithi094 ceph-mon[27862]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:54.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:54 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:54.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:54 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:54.832 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:54 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:54 smithi138 ceph-mon[32875]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:54 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:54 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:54 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:54 smithi137 ceph-mon[32439]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:54 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:54 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:54 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:57:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:56 smithi138 ceph-mon[32875]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:56.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:56 smithi094 ceph-mon[27862]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:56 smithi137 ceph-mon[32439]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:58.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.097+0000 7f25fbe00700 1 -- 172.21.15.94:0/1758338210 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2604062e80 con 0x7f25e80a3c40 2024-04-25T14:57:58.100 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 9 on host 'smithi094' 2024-04-25T14:57:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25e80a3c40 msgr2=0x7f25e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25e80a3c40 0x7f25e80a6100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25ec009c60 tx=0x7f25ec009260 comp rx=0 tx=0).stop 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 msgr2=0x7f260419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 0x7f260419ae50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25f400e3f0 tx=0x7f25f400e700 comp rx=0 tx=0).stop 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 shutdown_connections 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f25e80a3c40 0x7f25e80a6100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2604108790 0x7f26041a2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2604103530 0x7f260419ae50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.101+0000 7f2609600700 1 --2- 172.21.15.94:0/1758338210 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2604105e60 0x7f260419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:57:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.102+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 >> 172.21.15.94:0/1758338210 conn(0x7f26040fcf40 msgr2=0x7f2604102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:57:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.102+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 shutdown_connections 2024-04-25T14:57:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:57:58.102+0000 7f2609600700 1 -- 172.21.15.94:0/1758338210 wait complete. 2024-04-25T14:57:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:58 smithi138 ceph-mon[32875]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:58 smithi138 ceph-mon[32875]: from='osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:57:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:58 smithi138 ceph-mon[32875]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:57:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:58 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:58 smithi094 ceph-mon[27862]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:58 smithi094 ceph-mon[27862]: from='osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:57:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:58 smithi094 ceph-mon[27862]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:57:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:58 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:58 smithi137 ceph-mon[32439]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:57:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:58 smithi137 ceph-mon[32439]: from='osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:57:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:58 smithi137 ceph-mon[32439]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T14:57:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:58 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:57:58.806 DEBUG:teuthology.orchestra.run.smithi094:osd.9> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T14:57:58.813 INFO:tasks.cephadm:Deploying osd.10 on smithi137 with /dev/vg_nvme/lv_1... 2024-04-25T14:57:58.814 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:57:59.058 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:57:59.460 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:59 smithi137 ceph-mon[32439]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:57:59.460 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:59 smithi137 ceph-mon[32439]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T14:57:59.460 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:59 smithi137 ceph-mon[32439]: from='osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:57:59.460 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:59 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:57:59.460 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:57:59 smithi137 ceph-mon[32439]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:57:59.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:59 smithi094 ceph-mon[27862]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:57:59.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:59 smithi094 ceph-mon[27862]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T14:57:59.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:59 smithi094 ceph-mon[27862]: from='osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:57:59.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:59 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:57:59.655 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:57:59 smithi094 ceph-mon[27862]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:57:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:59 smithi138 ceph-mon[32875]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T14:57:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:59 smithi138 ceph-mon[32875]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T14:57:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:59 smithi138 ceph-mon[32875]: from='osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:57:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:59 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:57:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:57:59 smithi138 ceph-mon[32875]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-04-25T14:58:00.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-mon[27862]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:58:00.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-mon[27862]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:58:00.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:00.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:00.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-mon[27862]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:58:00.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-mon[27862]: from='osd.9 ' entity='osd.9' 2024-04-25T14:58:00.463 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T14:58:00.295+0000 7f9573000700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T14:58:00.463 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 14:58:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T14:58:00.322+0000 7f9568e00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:58:00.466 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:00 smithi137 ceph-mon[32439]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:58:00.466 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:00 smithi137 ceph-mon[32439]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:58:00.466 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:00.466 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:00 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:00.466 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:00 smithi137 ceph-mon[32439]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:58:00.466 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:00 smithi137 ceph-mon[32439]: from='osd.9 ' entity='osd.9' 2024-04-25T14:58:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:00 smithi138 ceph-mon[32875]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-04-25T14:58:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:00 smithi138 ceph-mon[32875]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T14:58:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:00 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:00 smithi138 ceph-mon[32875]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-25T14:58:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:00 smithi138 ceph-mon[32875]: from='osd.9 ' entity='osd.9' 2024-04-25T14:58:01.027 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:58:01.027 INFO:teuthology.orchestra.run.smithi137.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:58:01.027 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10+0 records in 2024-04-25T14:58:01.027 INFO:teuthology.orchestra.run.smithi137.stderr:10+0 records out 2024-04-25T14:58:01.027 INFO:teuthology.orchestra.run.smithi137.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126673 s, 828 MB/s 2024-04-25T14:58:01.027 INFO:teuthology.orchestra.run.smithi137.stderr:--> Zapping successful for: 2024-04-25T14:58:01.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:01 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:58:01.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:01 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:58:01.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:01.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:01 smithi094 ceph-mon[27862]: osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] boot 2024-04-25T14:58:01.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:01 smithi094 ceph-mon[27862]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T14:58:01.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:01 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:58:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:01 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:58:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:01 smithi138 ceph-mon[32875]: osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] boot 2024-04-25T14:58:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:01 smithi138 ceph-mon[32875]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T14:58:01.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:01.712 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:01 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:58:01.712 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:01 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:58:01.712 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:01.713 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:01 smithi137 ceph-mon[32439]: osd.9 [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] boot 2024-04-25T14:58:01.713 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:01 smithi137 ceph-mon[32439]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T14:58:01.713 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T14:58:01.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi137:vg_nvme/lv_1 2024-04-25T14:58:01.988 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T14:58:02.406 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:02 smithi137 ceph-mon[32439]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:02 smithi138 ceph-mon[32875]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:02.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:02 smithi094 ceph-mon[27862]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:03.408 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:03 smithi137 ceph-mon[32439]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T14:58:03.554 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:03 smithi094 ceph-mon[27862]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T14:58:03.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 -- 172.21.15.137:0/1212630160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 msgr2=0x7efd40101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:03.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 --2- 172.21.15.137:0/1212630160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40101de0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd300099c0 tx=0x7efd30009cd0 comp rx=0 tx=0).stop 2024-04-25T14:58:03.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 -- 172.21.15.137:0/1212630160 shutdown_connections 2024-04-25T14:58:03.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 --2- 172.21.15.137:0/1212630160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd40103560 0x7efd40109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:03.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 --2- 172.21.15.137:0/1212630160 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40101de0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:03.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 --2- 172.21.15.137:0/1212630160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd40102bc0 0x7efd40103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:03.660 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 -- 172.21.15.137:0/1212630160 >> 172.21.15.137:0/1212630160 conn(0x7efd400fcfa0 msgr2=0x7efd400ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:03.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 -- 172.21.15.137:0/1212630160 shutdown_connections 2024-04-25T14:58:03.661 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.659+0000 7efd44f31700 1 -- 172.21.15.137:0/1212630160 wait complete. 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 Processor -- start 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 -- start start 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd40102bc0 0x7efd40198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd40103560 0x7efd4019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efd4010d4b0 con 0x7efd40103560 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efd4010d330 con 0x7efd401019c0 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd44f31700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd4010d630 con 0x7efd40102bc0 2024-04-25T14:58:03.663 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.137:36736/0 (socket says 172.21.15.137:36736) 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 -- 172.21.15.137:0/1038090871 learned_addr learned my addr 172.21.15.137:0/1038090871 (peer_addr_for_me v2:172.21.15.137:0/0) 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3dd9b700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd40102bc0 0x7efd40198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 -- 172.21.15.137:0/1038090871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd40102bc0 msgr2=0x7efd40198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd40102bc0 0x7efd40198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 -- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd40103560 msgr2=0x7efd4019f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd40103560 0x7efd4019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3e59c700 1 -- 172.21.15.137:0/1038090871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4019f7b0 con 0x7efd401019c0 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.660+0000 7efd3dd9b700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd40102bc0 0x7efd40198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:58:03.664 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.661+0000 7efd3e59c700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40197b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efd300038c0 tx=0x7efd300047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:03.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.661+0000 7efd2f7fe700 1 -- 172.21.15.137:0/1038090871 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30027070 con 0x7efd401019c0 2024-04-25T14:58:03.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.661+0000 7efd2f7fe700 1 -- 172.21.15.137:0/1038090871 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd300041a0 con 0x7efd401019c0 2024-04-25T14:58:03.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.661+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4019fa40 con 0x7efd401019c0 2024-04-25T14:58:03.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.661+0000 7efd2f7fe700 1 -- 172.21.15.137:0/1038090871 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3000f460 con 0x7efd401019c0 2024-04-25T14:58:03.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.661+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4019ff20 con 0x7efd401019c0 2024-04-25T14:58:03.665 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.662+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd40069730 con 0x7efd401019c0 2024-04-25T14:58:03.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.663+0000 7efd2f7fe700 1 -- 172.21.15.137:0/1038090871 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efd3002a070 con 0x7efd401019c0 2024-04-25T14:58:03.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.663+0000 7efd2f7fe700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efd280a3c40 0x7efd280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:03.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.664+0000 7efd2f7fe700 1 -- 172.21.15.137:0/1038090871 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7efd300d0b50 con 0x7efd401019c0 2024-04-25T14:58:03.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.664+0000 7efd3dd9b700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efd280a3c40 0x7efd280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:03.666 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.665+0000 7efd3dd9b700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efd280a3c40 0x7efd280a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd34005fd0 tx=0x7efd34005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:03.668 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.668+0000 7efd2f7fe700 1 -- 172.21.15.137:0/1038090871 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd3005eff0 con 0x7efd401019c0 2024-04-25T14:58:03.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:03 smithi138 ceph-mon[32875]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T14:58:03.879 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:03.878+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efd40062e80 con 0x7efd280a3c40 2024-04-25T14:58:04.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:04.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:04.484 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:04 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:04.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:04 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:04.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:04.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:04.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:04 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:05.657 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:05 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi094 2024-04-25T14:58:05.657 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:05 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T14:58:05.657 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:05 smithi137 ceph-mon[32439]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:05.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:05 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi094 2024-04-25T14:58:05.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:05 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T14:58:05.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:05 smithi138 ceph-mon[32875]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:05.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:05 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi094 2024-04-25T14:58:05.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:05 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi094 to 4272M 2024-04-25T14:58:05.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:05 smithi094 ceph-mon[27862]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi137:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:06.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:06 smithi138 ceph-mon[32875]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:06.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:06 smithi094 ceph-mon[27862]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:06.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:06 smithi137 ceph-mon[32439]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:07 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/914907390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]: dispatch 2024-04-25T14:58:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:07 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]: dispatch 2024-04-25T14:58:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:07 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]': finished 2024-04-25T14:58:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:07 smithi138 ceph-mon[32875]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T14:58:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:07 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:07 smithi138 ceph-mon[32875]: from='client.? 172.21.15.137:0/2719110309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:07 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/914907390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]: dispatch 2024-04-25T14:58:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:07 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]: dispatch 2024-04-25T14:58:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:07 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]': finished 2024-04-25T14:58:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:07 smithi094 ceph-mon[27862]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T14:58:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:07 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:07 smithi094 ceph-mon[27862]: from='client.? 172.21.15.137:0/2719110309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:07 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/914907390' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]: dispatch 2024-04-25T14:58:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:07 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]: dispatch 2024-04-25T14:58:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:07 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a0daed0-84c8-4c9f-977e-d2502f39441e"}]': finished 2024-04-25T14:58:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:07 smithi137 ceph-mon[32439]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T14:58:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:07 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:07 smithi137 ceph-mon[32439]: from='client.? 172.21.15.137:0/2719110309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:08 smithi138 ceph-mon[32875]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:08 smithi094 ceph-mon[27862]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:08.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:08 smithi137 ceph-mon[32439]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:10.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:10 smithi137 ceph-mon[32439]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:10 smithi138 ceph-mon[32875]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:10 smithi094 ceph-mon[27862]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:12.844 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:12 smithi137 ceph-mon[32439]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:12 smithi138 ceph-mon[32875]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:12.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:12 smithi094 ceph-mon[27862]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:14.767 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:14 smithi137 ceph-mon[32439]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:14.767 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:58:14.767 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:14 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:14 smithi138 ceph-mon[32875]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:58:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:14 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:14 smithi094 ceph-mon[27862]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T14:58:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:14 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:15.771 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:15 smithi137 ceph-mon[32439]: Deploying daemon osd.10 on smithi137 2024-04-25T14:58:15.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:15 smithi138 ceph-mon[32875]: Deploying daemon osd.10 on smithi137 2024-04-25T14:58:15.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:15 smithi094 ceph-mon[27862]: Deploying daemon osd.10 on smithi137 2024-04-25T14:58:16.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:16 smithi137 ceph-mon[32439]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:16 smithi138 ceph-mon[32875]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:16 smithi094 ceph-mon[27862]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:18.863 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:18 smithi137 ceph-mon[32439]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:18.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:18 smithi138 ceph-mon[32875]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:18 smithi094 ceph-mon[27862]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:20.695 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:20 smithi137 ceph-mon[32439]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:20.695 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:20.695 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:20.695 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:20 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:20.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:20 smithi138 ceph-mon[32875]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:20.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:20.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:20.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:20 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:20 smithi094 ceph-mon[27862]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:20 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:22 smithi138 ceph-mon[32875]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:22 smithi094 ceph-mon[27862]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:22 smithi137 ceph-mon[32439]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:23.560 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.558+0000 7efd2f7fe700 1 -- 172.21.15.137:0/1038090871 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efd40062e80 con 0x7efd280a3c40 2024-04-25T14:58:23.560 INFO:teuthology.orchestra.run.smithi137.stdout:Created osd(s) 10 on host 'smithi137' 2024-04-25T14:58:23.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efd280a3c40 msgr2=0x7efd280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:23.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efd280a3c40 0x7efd280a6100 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd34005fd0 tx=0x7efd34005f60 comp rx=0 tx=0).stop 2024-04-25T14:58:23.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 msgr2=0x7efd40197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:23.562 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40197b50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efd300038c0 tx=0x7efd300047b0 comp rx=0 tx=0).stop 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 shutdown_connections 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efd280a3c40 0x7efd280a6100 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd40103560 0x7efd4019f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efd401019c0 0x7efd40197b50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 --2- 172.21.15.137:0/1038090871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd40102bc0 0x7efd40198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.560+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 >> 172.21.15.137:0/1038090871 conn(0x7efd400fcfa0 msgr2=0x7efd40105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.561+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 shutdown_connections 2024-04-25T14:58:23.563 INFO:teuthology.orchestra.run.smithi137.stderr:2024-04-25T14:58:23.561+0000 7efd44f31700 1 -- 172.21.15.137:0/1038090871 wait complete. 2024-04-25T14:58:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:23.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:23.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:23.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:58:23.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:23 smithi137 ceph-mon[32439]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:58:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:23 smithi138 ceph-mon[32875]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:58:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:23.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:23.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:58:23.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:23 smithi094 ceph-mon[27862]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T14:58:24.153 DEBUG:teuthology.orchestra.run.smithi137:osd.10> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T14:58:24.156 INFO:tasks.cephadm:Deploying osd.11 on smithi138 with /dev/vg_nvme/lv_1... 2024-04-25T14:58:24.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T14:58:24.403 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:58:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:24 smithi137 ceph-mon[32439]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:24 smithi137 ceph-mon[32439]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:58:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:24 smithi137 ceph-mon[32439]: from='osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:58:24.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:24 smithi137 ceph-mon[32439]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T14:58:24.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:24.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:24 smithi137 ceph-mon[32439]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:58:24.842 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:24 smithi138 ceph-mon[32875]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:24.843 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:24 smithi138 ceph-mon[32875]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:58:24.843 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:24 smithi138 ceph-mon[32875]: from='osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:58:24.843 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:24 smithi138 ceph-mon[32875]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T14:58:24.843 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:24.843 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:24 smithi138 ceph-mon[32875]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:58:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:24 smithi094 ceph-mon[27862]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:24 smithi094 ceph-mon[27862]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T14:58:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:24 smithi094 ceph-mon[27862]: from='osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:58:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:24 smithi094 ceph-mon[27862]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T14:58:24.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:24.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:24 smithi094 ceph-mon[27862]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]: dispatch 2024-04-25T14:58:25.982 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:58:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T14:58:25.520+0000 7fd959986700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T14:58:25.983 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 14:58:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T14:58:25.533+0000 7fd955779700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:58:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T14:58:26.447 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T14:58:26.448 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T14:58:26.448 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T14:58:26.448 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012878 s, 814 MB/s 2024-04-25T14:58:26.448 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T14:58:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:26 smithi137 ceph-mon[32439]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:26 smithi137 ceph-mon[32439]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:58:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:26 smithi137 ceph-mon[32439]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T14:58:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:26.642 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:26 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:26.745 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:26 smithi138 ceph-mon[32875]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:26.745 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:26 smithi138 ceph-mon[32875]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:58:26.745 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:26 smithi138 ceph-mon[32875]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T14:58:26.745 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:26.745 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:26 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:26.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:26 smithi094 ceph-mon[27862]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-04-25T14:58:26.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:26 smithi094 ceph-mon[27862]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi137", "root=default"]}]': finished 2024-04-25T14:58:26.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:26 smithi094 ceph-mon[27862]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T14:58:26.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:26.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:26 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:27.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-04-25T14:58:27.294 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:58:27.699 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:58:27.699 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:58:27.699 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] boot 2024-04-25T14:58:27.699 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T14:58:27.699 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:27.699 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:27.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] boot 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:27.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:27.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:58:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:58:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: osd.10 [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] boot 2024-04-25T14:58:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T14:58:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T14:58:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:27.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:28.579 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:28 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi137 2024-04-25T14:58:28.579 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:28 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-25T14:58:28.579 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:28 smithi138 ceph-mon[32875]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:28.579 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:28 smithi138 ceph-mon[32875]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 -- 172.21.15.138:0/4096582383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 msgr2=0x7eff8c1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 --2- 172.21.15.138:0/4096582383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c1020c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7eff740099c0 tx=0x7eff74009cd0 comp rx=0 tx=0).stop 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 -- 172.21.15.138:0/4096582383 shutdown_connections 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 --2- 172.21.15.138:0/4096582383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8c102600 0x7eff8c109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 --2- 172.21.15.138:0/4096582383 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff8c102f20 0x7eff8c103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 --2- 172.21.15.138:0/4096582383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c1020c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 -- 172.21.15.138:0/4096582383 >> 172.21.15.138:0/4096582383 conn(0x7eff8c0fc6d0 msgr2=0x7eff8c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:28.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 -- 172.21.15.138:0/4096582383 shutdown_connections 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 -- 172.21.15.138:0/4096582383 wait complete. 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.816+0000 7eff9169f700 1 Processor -- start 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff9169f700 1 -- start start 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff9169f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c105b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff9169f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8c102600 0x7eff8c1041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff9169f700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff8c102f20 0x7eff8c1046e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff9169f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10d480 con 0x7eff8c102600 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff9169f700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10d300 con 0x7eff8c102f20 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff9169f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c10d600 con 0x7eff8c101c60 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c105b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c105b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38142/0 (socket says 172.21.15.138:38142) 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 -- 172.21.15.138:0/2444686894 learned_addr learned my addr 172.21.15.138:0/2444686894 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:58:28.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8b7fe700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff8c102f20 0x7eff8c1046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 -- 172.21.15.138:0/2444686894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff8c102f20 msgr2=0x7eff8c1046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff8c102f20 0x7eff8c1046e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 -- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8c102600 msgr2=0x7eff8c1041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8c102600 0x7eff8c1041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 -- 172.21.15.138:0/2444686894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c104f90 con 0x7eff8c101c60 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff8affd700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c105b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff84007e00 tx=0x7eff8400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.817+0000 7eff83fff700 1 -- 172.21.15.138:0/2444686894 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff84016020 con 0x7eff8c101c60 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.818+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c1a2d10 con 0x7eff8c101c60 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.818+0000 7eff83fff700 1 -- 172.21.15.138:0/2444686894 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff8400ad40 con 0x7eff8c101c60 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.818+0000 7eff83fff700 1 -- 172.21.15.138:0/2444686894 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff840115e0 con 0x7eff8c101c60 2024-04-25T14:58:28.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.818+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c1a31a0 con 0x7eff8c101c60 2024-04-25T14:58:28.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.819+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c1a3430 con 0x7eff8c101c60 2024-04-25T14:58:28.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.820+0000 7eff83fff700 1 -- 172.21.15.138:0/2444686894 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7eff84019020 con 0x7eff8c101c60 2024-04-25T14:58:28.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.820+0000 7eff83fff700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7eff780a3c40 0x7eff780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:28.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.820+0000 7eff83fff700 1 -- 172.21.15.138:0/2444686894 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7eff840cfac0 con 0x7eff8c101c60 2024-04-25T14:58:28.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.821+0000 7eff8a7fc700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7eff780a3c40 0x7eff780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:28.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.821+0000 7eff8a7fc700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7eff780a3c40 0x7eff780a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff74003790 tx=0x7eff74009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:28.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:28.825+0000 7eff83fff700 1 -- 172.21.15.138:0/2444686894 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff84020030 con 0x7eff8c101c60 2024-04-25T14:58:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:28 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi137 2024-04-25T14:58:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:28 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-25T14:58:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:28 smithi094 ceph-mon[27862]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:28 smithi094 ceph-mon[27862]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T14:58:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:28 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi137 2024-04-25T14:58:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:28 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi137 to 4272M 2024-04-25T14:58:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:28 smithi137 ceph-mon[32439]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:28 smithi137 ceph-mon[32439]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T14:58:29.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:29.037+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7eff8c1a3600 con 0x7eff780a3c40 2024-04-25T14:58:30.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:29 smithi138 ceph-mon[32875]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T14:58:30.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:29 smithi138 ceph-mon[32875]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:30.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:30.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:30.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:30.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:29 smithi138 ceph-mon[32875]: pgmap v260: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-25T14:58:30.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:29 smithi094 ceph-mon[27862]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T14:58:30.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:29 smithi094 ceph-mon[27862]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:30.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:30.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:30.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:30.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:29 smithi094 ceph-mon[27862]: pgmap v260: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-25T14:58:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:29 smithi137 ceph-mon[32439]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T14:58:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:29 smithi137 ceph-mon[32439]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:58:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T14:58:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T14:58:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:29 smithi137 ceph-mon[32439]: pgmap v260: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-04-25T14:58:30.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:30 smithi138 ceph-mon[32875]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T14:58:31.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:30 smithi094 ceph-mon[27862]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T14:58:31.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:30 smithi137 ceph-mon[32439]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T14:58:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:31 smithi138 ceph-mon[32875]: pgmap v262: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-25T14:58:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:31 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/3871512827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]: dispatch 2024-04-25T14:58:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:31 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]: dispatch 2024-04-25T14:58:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:31 smithi138 ceph-mon[32875]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]': finished 2024-04-25T14:58:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:31 smithi138 ceph-mon[32875]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T14:58:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:31 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:31 smithi094 ceph-mon[27862]: pgmap v262: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-25T14:58:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:31 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/3871512827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]: dispatch 2024-04-25T14:58:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:31 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]: dispatch 2024-04-25T14:58:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:31 smithi094 ceph-mon[27862]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]': finished 2024-04-25T14:58:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:31 smithi094 ceph-mon[27862]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T14:58:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:31 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:31 smithi137 ceph-mon[32439]: pgmap v262: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-25T14:58:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:31 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/3871512827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]: dispatch 2024-04-25T14:58:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:31 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]: dispatch 2024-04-25T14:58:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:31 smithi137 ceph-mon[32439]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b938f69-8356-40c2-9a76-45bed8d6a261"}]': finished 2024-04-25T14:58:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:31 smithi137 ceph-mon[32439]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T14:58:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:31 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:33.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:32 smithi138 ceph-mon[32875]: from='client.? 172.21.15.138:0/3750929011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:33.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:32 smithi094 ceph-mon[27862]: from='client.? 172.21.15.138:0/3750929011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:33.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:32 smithi137 ceph-mon[32439]: from='client.? 172.21.15.138:0/3750929011' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T14:58:34.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:33 smithi094 ceph-mon[27862]: pgmap v264: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T14:58:34.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:33 smithi137 ceph-mon[32439]: pgmap v264: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T14:58:34.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:33 smithi138 ceph-mon[32875]: pgmap v264: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-04-25T14:58:36.794 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:36 smithi138 ceph-mon[32875]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:36.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:36 smithi094 ceph-mon[27862]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:36 smithi137 ceph-mon[32439]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:38.834 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:38 smithi138 ceph-mon[32875]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:38 smithi094 ceph-mon[27862]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:38 smithi137 ceph-mon[32439]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:39.786 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:39 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:58:39.786 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:39 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:39 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:58:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:39 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:39.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:39 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T14:58:39.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:39 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:40.566 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:40 smithi138 ceph-mon[32875]: Deploying daemon osd.11 on smithi138 2024-04-25T14:58:40.566 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:40 smithi138 ceph-mon[32875]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:40.864 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:40 smithi094 ceph-mon[27862]: Deploying daemon osd.11 on smithi138 2024-04-25T14:58:40.864 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:40 smithi094 ceph-mon[27862]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:40 smithi137 ceph-mon[32439]: Deploying daemon osd.11 on smithi138 2024-04-25T14:58:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:40 smithi137 ceph-mon[32439]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:42.829 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:42 smithi138 ceph-mon[32875]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:42.832 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:42 smithi137 ceph-mon[32439]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:42.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:42 smithi094 ceph-mon[27862]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:44.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:44 smithi138 ceph-mon[32875]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:44 smithi094 ceph-mon[27862]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:44.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:44 smithi137 ceph-mon[32439]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:45.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:45.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:45.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:45.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:46.849 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:46 smithi138 ceph-mon[32875]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:46.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:46 smithi094 ceph-mon[27862]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:46 smithi137 ceph-mon[32439]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:48.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.121+0000 7eff83fff700 1 -- 172.21.15.138:0/2444686894 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7eff8c1a3600 con 0x7eff780a3c40 2024-04-25T14:58:48.123 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 11 on host 'smithi138' 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7eff780a3c40 msgr2=0x7eff780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7eff780a3c40 0x7eff780a6100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7eff74003790 tx=0x7eff74009f10 comp rx=0 tx=0).stop 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 msgr2=0x7eff8c105b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c105b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7eff84007e00 tx=0x7eff8400a340 comp rx=0 tx=0).stop 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 shutdown_connections 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7eff780a3c40 0x7eff780a6100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7eff8c102f20 0x7eff8c1046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff8c101c60 0x7eff8c105b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:48.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 --2- 172.21.15.138:0/2444686894 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7eff8c102600 0x7eff8c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:48.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 >> 172.21.15.138:0/2444686894 conn(0x7eff8c0fc6d0 msgr2=0x7eff8c100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:48.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 shutdown_connections 2024-04-25T14:58:48.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:58:48.124+0000 7eff9169f700 1 -- 172.21.15.138:0/2444686894 wait complete. 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.678 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:48.703 DEBUG:teuthology.orchestra.run.smithi138:osd.11> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T14:58:48.705 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T14:58:48.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T14:58:48.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:48.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:48.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:48.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:48.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:48.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:58:49.145 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:58:49.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:49 smithi094 ceph-mon[27862]: from='osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:58:49.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:49 smithi094 ceph-mon[27862]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:58:49.700 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:49 smithi138 ceph-mon[32875]: from='osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:58:49.701 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:49 smithi138 ceph-mon[32875]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:58:49.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:49 smithi137 ceph-mon[32439]: from='osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:58:49.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:49 smithi137 ceph-mon[32439]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T14:58:50.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:50 smithi094 ceph-mon[27862]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:58:50.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:50 smithi094 ceph-mon[27862]: from='osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:58:50.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:50 smithi094 ceph-mon[27862]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T14:58:50.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:50 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:50.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:50 smithi094 ceph-mon[27862]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:58:50.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:50 smithi094 ceph-mon[27862]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:50.587 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:50 smithi094 ceph-mon[27862]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:58:50.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-mon[32875]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:58:50.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-mon[32875]: from='osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:58:50.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-mon[32875]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T14:58:50.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:50.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-mon[32875]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:58:50.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-mon[32875]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:50.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-mon[32875]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:58:50.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T14:58:50.355+0000 7f4511dee700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T14:58:50.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 14:58:50 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T14:58:50.370+0000 7f450dbe1700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T14:58:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:50 smithi137 ceph-mon[32439]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T14:58:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:50 smithi137 ceph-mon[32439]: from='osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:58:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:50 smithi137 ceph-mon[32439]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T14:58:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:50 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:50 smithi137 ceph-mon[32439]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T14:58:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:50 smithi137 ceph-mon[32439]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:50.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:50 smithi137 ceph-mon[32439]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.349+0000 7f3eb6600700 1 -- 172.21.15.94:0/2820275863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 msgr2=0x7f3eb0109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.349+0000 7f3eb6600700 1 --2- 172.21.15.94:0/2820275863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 0x7f3eb0109500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000b0a0 tx=0x7f3ea000b3b0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.350+0000 7f3eb6600700 1 -- 172.21.15.94:0/2820275863 shutdown_connections 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.350+0000 7f3eb6600700 1 --2- 172.21.15.94:0/2820275863 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010ac80 0x7f3eb0111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.350+0000 7f3eb6600700 1 --2- 172.21.15.94:0/2820275863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 0x7f3eb0109500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.350+0000 7f3eb6600700 1 --2- 172.21.15.94:0/2820275863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eb010a2e0 0x7f3eb010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.350+0000 7f3eb6600700 1 -- 172.21.15.94:0/2820275863 >> 172.21.15.94:0/2820275863 conn(0x7f3eb00780b0 msgr2=0x7f3eb007a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.350+0000 7f3eb6600700 1 -- 172.21.15.94:0/2820275863 shutdown_connections 2024-04-25T14:58:51.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.350+0000 7f3eb6600700 1 -- 172.21.15.94:0/2820275863 wait complete. 2024-04-25T14:58:51.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 Processor -- start 2024-04-25T14:58:51.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 -- start start 2024-04-25T14:58:51.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 0x7f3eb019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:51.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010a2e0 0x7f3eb019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eb010ac80 0x7f3eb01a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0114ba0 con 0x7f3eb010ac80 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0114a20 con 0x7f3eb010a2e0 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.351+0000 7f3eb6600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0114d20 con 0x7f3eb01090e0 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010a2e0 0x7f3eb019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3eafe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eb010ac80 0x7f3eb01a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3eafe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eb010ac80 0x7f3eb01a67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52714/0 (socket says 172.21.15.94:52714) 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010a2e0 0x7f3eb019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38670/0 (socket says 172.21.15.94:38670) 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3eafe00700 1 -- 172.21.15.94:0/110177164 learned_addr learned my addr 172.21.15.94:0/110177164 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:58:51.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3eaf400700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 0x7f3eb019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 -- 172.21.15.94:0/110177164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 msgr2=0x7f3eb019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 0x7f3eb019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 -- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eb010ac80 msgr2=0x7f3eb01a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eb010ac80 0x7f3eb01a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 -- 172.21.15.94:0/110177164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb01a6e70 con 0x7f3eb010a2e0 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.352+0000 7f3ea6a00700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010a2e0 0x7f3eb019f7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e9f0 tx=0x7f3e9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.353+0000 7f3eacc00700 1 -- 172.21.15.94:0/110177164 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800cbd0 con 0x7f3eb010a2e0 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.353+0000 7f3eacc00700 1 -- 172.21.15.94:0/110177164 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e9800cd70 con 0x7f3eb010a2e0 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.353+0000 7f3eacc00700 1 -- 172.21.15.94:0/110177164 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98018660 con 0x7f3eb010a2e0 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.353+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb01a7160 con 0x7f3eb010a2e0 2024-04-25T14:58:51.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.353+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb01a7650 con 0x7f3eb010a2e0 2024-04-25T14:58:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.354+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0069730 con 0x7f3eb010a2e0 2024-04-25T14:58:51.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.356+0000 7f3eacc00700 1 -- 172.21.15.94:0/110177164 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3e98010430 con 0x7f3eb010a2e0 2024-04-25T14:58:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.357+0000 7f3eacc00700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e940a3cc0 0x7f3e940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.358+0000 7f3eacc00700 1 -- 172.21.15.94:0/110177164 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f3e980cffe0 con 0x7f3eb010a2e0 2024-04-25T14:58:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.358+0000 7f3eaf400700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e940a3cc0 0x7f3e940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:51.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.358+0000 7f3eaf400700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e940a3cc0 0x7f3e940a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009fd0 tx=0x7f3ea0006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:51.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.362+0000 7f3eacc00700 1 -- 172.21.15.94:0/110177164 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e9809cf00 con 0x7f3eb010a2e0 2024-04-25T14:58:51.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:51 smithi094 ceph-mon[27862]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T14:58:51.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:51 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:51.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:51 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:51.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.613+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3eb0063650 con 0x7f3eb010a2e0 2024-04-25T14:58:51.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.614+0000 7f3eacc00700 1 -- 172.21.15.94:0/110177164 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f3e9809c800 con 0x7f3eb010a2e0 2024-04-25T14:58:51.614 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:58:51.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e940a3cc0 msgr2=0x7f3e940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:51.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e940a3cc0 0x7f3e940a6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0009fd0 tx=0x7f3ea0006000 comp rx=0 tx=0).stop 2024-04-25T14:58:51.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010a2e0 msgr2=0x7f3eb019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010a2e0 0x7f3eb019f7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e9f0 tx=0x7f3e9800ed00 comp rx=0 tx=0).stop 2024-04-25T14:58:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 shutdown_connections 2024-04-25T14:58:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3e940a3cc0 0x7f3e940a6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3eb010ac80 0x7f3eb01a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eb01090e0 0x7f3eb019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 --2- 172.21.15.94:0/110177164 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3eb010a2e0 0x7f3eb019f7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.616+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 >> 172.21.15.94:0/110177164 conn(0x7f3eb00780b0 msgr2=0x7f3eb010d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.617+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 shutdown_connections 2024-04-25T14:58:51.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:51.617+0000 7f3eb6600700 1 -- 172.21.15.94:0/110177164 wait complete. 2024-04-25T14:58:51.706 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:51 smithi138 ceph-mon[32875]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T14:58:51.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:51 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:51.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:51 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:51.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:51 smithi137 ceph-mon[32439]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T14:58:51.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:51 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:51.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:51 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:52.442 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714057131,"num_in_osds":12,"osd_in_since":1714057111,"num_remapped_pgs":0} 2024-04-25T14:58:52.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T14:58:52.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: purged_snaps scrub starts 2024-04-25T14:58:52.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: purged_snaps scrub ok 2024-04-25T14:58:52.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] boot 2024-04-25T14:58:52.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: Detected new or changed devices on smithi138 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:52.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.709 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:52 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/110177164' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: purged_snaps scrub starts 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: purged_snaps scrub ok 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] boot 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: Detected new or changed devices on smithi138 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:52 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/110177164' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:58:52.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: purged_snaps scrub starts 2024-04-25T14:58:52.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: purged_snaps scrub ok 2024-04-25T14:58:52.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: osd.11 [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] boot 2024-04-25T14:58:52.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T14:58:52.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: Detected new or changed devices on smithi138 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi138 to 4272M 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:58:52.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:52 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/110177164' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T14:58:52.876 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:58:53.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:53 smithi138 ceph-mon[32875]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T14:58:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:53 smithi094 ceph-mon[27862]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T14:58:53.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:53 smithi137 ceph-mon[32439]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T14:58:54.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:54 smithi094 ceph-mon[27862]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:54.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:54 smithi137 ceph-mon[32439]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:54.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.931+0000 7f0eb3600700 1 -- 172.21.15.94:0/3023037531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 msgr2=0x7f0eac10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:54.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.931+0000 7f0eb3600700 1 --2- 172.21.15.94:0/3023037531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac10f8b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400b0a0 tx=0x7f0ea400b3b0 comp rx=0 tx=0).stop 2024-04-25T14:58:54.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.932+0000 7f0eb3600700 1 -- 172.21.15.94:0/3023037531 shutdown_connections 2024-04-25T14:58:54.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.932+0000 7f0eb3600700 1 --2- 172.21.15.94:0/3023037531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac10f8b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:54.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.932+0000 7f0eb3600700 1 --2- 172.21.15.94:0/3023037531 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eac108a50 0x7f0eac108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:54.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.932+0000 7f0eb3600700 1 --2- 172.21.15.94:0/3023037531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac102a90 0x7f0eac102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:54.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.932+0000 7f0eb3600700 1 -- 172.21.15.94:0/3023037531 >> 172.21.15.94:0/3023037531 conn(0x7f0eac079ba0 msgr2=0x7f0eac079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:54.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.932+0000 7f0eb3600700 1 -- 172.21.15.94:0/3023037531 shutdown_connections 2024-04-25T14:58:54.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.932+0000 7f0eb3600700 1 -- 172.21.15.94:0/3023037531 wait complete. 2024-04-25T14:58:54.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 Processor -- start 2024-04-25T14:58:54.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 -- start start 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eac102a90 0x7f0eac19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac108a50 0x7f0eac19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac112c50 con 0x7f0eac103450 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac112ad0 con 0x7f0eac102a90 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.933+0000 7f0eb3600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac112dd0 con 0x7f0eac108a50 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eabe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eabe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac19b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52730/0 (socket says 172.21.15.94:52730) 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eb1600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac108a50 0x7f0eac19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eabe00700 1 -- 172.21.15.94:0/2807349981 learned_addr learned my addr 172.21.15.94:0/2807349981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eb0c00700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eac102a90 0x7f0eac19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eb1600700 1 -- 172.21.15.94:0/2807349981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eac102a90 msgr2=0x7f0eac19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eb1600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eac102a90 0x7f0eac19b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:54.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eb1600700 1 -- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 msgr2=0x7f0eac19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:54.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eb1600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:54.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eb1600700 1 -- 172.21.15.94:0/2807349981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac1a0330 con 0x7f0eac108a50 2024-04-25T14:58:54.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.934+0000 7f0eabe00700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:58:54.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.935+0000 7f0eb1600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac108a50 0x7f0eac19fbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40062a0 tx=0x7f0ea4004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:54.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.935+0000 7f0ea9600700 1 -- 172.21.15.94:0/2807349981 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4021070 con 0x7f0eac108a50 2024-04-25T14:58:54.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.935+0000 7f0ea9600700 1 -- 172.21.15.94:0/2807349981 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea4007b80 con 0x7f0eac108a50 2024-04-25T14:58:54.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.935+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac1a05c0 con 0x7f0eac108a50 2024-04-25T14:58:54.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.935+0000 7f0ea9600700 1 -- 172.21.15.94:0/2807349981 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea401de60 con 0x7f0eac108a50 2024-04-25T14:58:54.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.935+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac1a7ea0 con 0x7f0eac108a50 2024-04-25T14:58:54.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.936+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac069730 con 0x7f0eac108a50 2024-04-25T14:58:54.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.938+0000 7f0ea9600700 1 -- 172.21.15.94:0/2807349981 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ea4024070 con 0x7f0eac108a50 2024-04-25T14:58:54.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.939+0000 7f0ea9600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e940a3c40 0x7f0e940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:54.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.940+0000 7f0ea9600700 1 -- 172.21.15.94:0/2807349981 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0ea400f020 con 0x7f0eac108a50 2024-04-25T14:58:54.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.940+0000 7f0eb0c00700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e940a3c40 0x7f0e940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:54.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.940+0000 7f0eb0c00700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e940a3c40 0x7f0e940a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005950 tx=0x7f0ea000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:54.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:54.944+0000 7f0ea9600700 1 -- 172.21.15.94:0/2807349981 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ea402b020 con 0x7f0eac108a50 2024-04-25T14:58:54.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:54 smithi138 ceph-mon[32875]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.170+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0eac1a8070 con 0x7f0eac108a50 2024-04-25T14:58:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.172+0000 7f0ea9600700 1 -- 172.21.15.94:0/2807349981 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18441 (secure 0 0 0) 0x7f0ea409e2a0 con 0x7f0eac108a50 2024-04-25T14:58:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:58:55.174 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T14:58:52.443830+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e940a3c40 msgr2=0x7f0e940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e940a3c40 0x7f0e940a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0005950 tx=0x7f0ea000b410 comp rx=0 tx=0).stop 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac108a50 msgr2=0x7f0eac19fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac108a50 0x7f0eac19fbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40062a0 tx=0x7f0ea4004cf0 comp rx=0 tx=0).stop 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 shutdown_connections 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e940a3c40 0x7f0e940a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eac108a50 0x7f0eac19fbf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eac102a90 0x7f0eac19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 --2- 172.21.15.94:0/2807349981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eac103450 0x7f0eac19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.175+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 >> 172.21.15.94:0/2807349981 conn(0x7f0eac079ba0 msgr2=0x7f0eac10de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.176+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 shutdown_connections 2024-04-25T14:58:55.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:55.176+0000 7f0eb3600700 1 -- 172.21.15.94:0/2807349981 wait complete. 2024-04-25T14:58:55.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:55 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2807349981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:58:55.843 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:55 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2807349981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:58:55.892 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:58:55.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T14:58:55.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2807349981' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:58:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:58:56.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:56 smithi137 ceph-mon[32439]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:56.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:56 smithi094 ceph-mon[27862]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:56 smithi138 ceph-mon[32875]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:58.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 -- 172.21.15.94:0/1209969861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e54102b00 msgr2=0x7f4e54102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 --2- 172.21.15.94:0/1209969861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e54102b00 0x7f4e54102f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4e40009990 tx=0x7f4e40009ca0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 -- 172.21.15.94:0/1209969861 shutdown_connections 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 --2- 172.21.15.94:0/1209969861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e541034c0 0x7f4e5410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 --2- 172.21.15.94:0/1209969861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e54108a10 0x7f4e54108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 --2- 172.21.15.94:0/1209969861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e54102b00 0x7f4e54102f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 -- 172.21.15.94:0/1209969861 >> 172.21.15.94:0/1209969861 conn(0x7f4e54079c00 msgr2=0x7f4e5407a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 -- 172.21.15.94:0/1209969861 shutdown_connections 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.441+0000 7f4e59000700 1 -- 172.21.15.94:0/1209969861 wait complete. 2024-04-25T14:58:58.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.442+0000 7f4e59000700 1 Processor -- start 2024-04-25T14:58:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e59000700 1 -- start start 2024-04-25T14:58:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e59000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54102b00 0x7f4e5419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e59000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e541034c0 0x7f4e5419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e59000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e54108a10 0x7f4e5419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e59000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112ca0 con 0x7f4e541034c0 2024-04-25T14:58:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e59000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112b20 con 0x7f4e54108a10 2024-04-25T14:58:58.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e59000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112e20 con 0x7f4e54102b00 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e51400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e541034c0 0x7f4e5419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e51400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e541034c0 0x7f4e5419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39036/0 (socket says 172.21.15.94:39036) 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e51400700 1 -- 172.21.15.94:0/1391034450 learned_addr learned my addr 172.21.15.94:0/1391034450 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e52800700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e54108a10 0x7f4e5419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.443+0000 7f4e51e00700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54102b00 0x7f4e5419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e51400700 1 -- 172.21.15.94:0/1391034450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54102b00 msgr2=0x7f4e5419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e51400700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54102b00 0x7f4e5419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e51400700 1 -- 172.21.15.94:0/1391034450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e54108a10 msgr2=0x7f4e5419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e51400700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e54108a10 0x7f4e5419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e51400700 1 -- 172.21.15.94:0/1391034450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e541a0370 con 0x7f4e541034c0 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e52800700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e54108a10 0x7f4e5419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e51400700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e541034c0 0x7f4e5419b8a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e4aa00700 1 -- 172.21.15.94:0/1391034450 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40027070 con 0x7f4e541034c0 2024-04-25T14:58:58.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e541a0600 con 0x7f4e541034c0 2024-04-25T14:58:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e4aa00700 1 -- 172.21.15.94:0/1391034450 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e40022470 con 0x7f4e541034c0 2024-04-25T14:58:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.444+0000 7f4e4aa00700 1 -- 172.21.15.94:0/1391034450 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40017650 con 0x7f4e541034c0 2024-04-25T14:58:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.445+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541a7ee0 con 0x7f4e541034c0 2024-04-25T14:58:58.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.446+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e54069730 con 0x7f4e541034c0 2024-04-25T14:58:58.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.448+0000 7f4e4aa00700 1 -- 172.21.15.94:0/1391034450 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e4002a070 con 0x7f4e541034c0 2024-04-25T14:58:58.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.449+0000 7f4e4aa00700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e380a3c40 0x7f4e380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:58:58.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.449+0000 7f4e4aa00700 1 -- 172.21.15.94:0/1391034450 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4e400d0da0 con 0x7f4e541034c0 2024-04-25T14:58:58.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.449+0000 7f4e51e00700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e380a3c40 0x7f4e380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:58:58.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.450+0000 7f4e51e00700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e380a3c40 0x7f4e380a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009be0 tx=0x7f4e44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:58:58.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.452+0000 7f4e4aa00700 1 -- 172.21.15.94:0/1391034450 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e40013070 con 0x7f4e541034c0 2024-04-25T14:58:58.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.680+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e541a80b0 con 0x7f4e541034c0 2024-04-25T14:58:58.682 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:58 smithi094 ceph-mon[27862]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:58.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.681+0000 7f4e4aa00700 1 -- 172.21.15.94:0/1391034450 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f4e4009dc10 con 0x7f4e541034c0 2024-04-25T14:58:58.683 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T14:58:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e380a3c40 msgr2=0x7f4e380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:58.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e380a3c40 0x7f4e380a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009be0 tx=0x7f4e44009260 comp rx=0 tx=0).stop 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e541034c0 msgr2=0x7f4e5419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e541034c0 0x7f4e5419b8a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400057c0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 shutdown_connections 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e380a3c40 0x7f4e380a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e54108a10 0x7f4e5419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e54102b00 0x7f4e5419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 --2- 172.21.15.94:0/1391034450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e541034c0 0x7f4e5419b8a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.685+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 >> 172.21.15.94:0/1391034450 conn(0x7f4e54079c00 msgr2=0x7f4e54100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.686+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 shutdown_connections 2024-04-25T14:58:58.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:58:58.686+0000 7f4e59000700 1 -- 172.21.15.94:0/1391034450 wait complete. 2024-04-25T14:58:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:58 smithi137 ceph-mon[32439]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:58.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:58 smithi138 ceph-mon[32875]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:58:59.432 INFO:tasks.cephadm:Adding mds.a on smithi094 2024-04-25T14:58:59.432 INFO:tasks.cephadm:Adding mds.d on smithi094 2024-04-25T14:58:59.432 INFO:tasks.cephadm:Adding mds.g on smithi094 2024-04-25T14:58:59.432 INFO:tasks.cephadm:Adding mds.j on smithi094 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.b on smithi137 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.e on smithi137 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.h on smithi137 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.k on smithi137 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.c on smithi138 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.f on smithi138 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.i on smithi138 2024-04-25T14:58:59.433 INFO:tasks.cephadm:Adding mds.l on smithi138 2024-04-25T14:58:59.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l' 2024-04-25T14:58:59.685 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T14:58:59.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:58:59 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1391034450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:58:59.807 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:58:59 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1391034450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:58:59.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:58:59 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1391034450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:59:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:00 smithi138 ceph-mon[32875]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:00 smithi094 ceph-mon[27862]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:00.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:00 smithi137 ceph-mon[32439]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:01.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 -- 172.21.15.138:0/191848529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 msgr2=0x7f4154102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 --2- 172.21.15.138:0/191848529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f4154102ec0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4144009990 tx=0x7f4144009ca0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 -- 172.21.15.138:0/191848529 shutdown_connections 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 --2- 172.21.15.138:0/191848529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4154104640 0x7f415410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 --2- 172.21.15.138:0/191848529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f4154102ec0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 --2- 172.21.15.138:0/191848529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4154103ca0 0x7f4154104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 -- 172.21.15.138:0/191848529 >> 172.21.15.138:0/191848529 conn(0x7f41540fe040 msgr2=0x7f4154100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 -- 172.21.15.138:0/191848529 shutdown_connections 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 -- 172.21.15.138:0/191848529 wait complete. 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 Processor -- start 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.348+0000 7f4158ad4700 1 -- start start 2024-04-25T14:59:01.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f415419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4154103ca0 0x7f415419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4154104640 0x7f41541a2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f415410e590 con 0x7f4154104640 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f415410e410 con 0x7f4154103ca0 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f415410e710 con 0x7f4154102aa0 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f415419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f415419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:45602/0 (socket says 172.21.15.138:45602) 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 -- 172.21.15.138:0/2268768224 learned_addr learned my addr 172.21.15.138:0/2268768224 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4151d9b700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4154103ca0 0x7f415419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 -- 172.21.15.138:0/2268768224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4154103ca0 msgr2=0x7f415419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4154103ca0 0x7f415419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 -- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4154104640 msgr2=0x7f41541a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4154104640 0x7f41541a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 -- 172.21.15.138:0/2268768224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41541a2ae0 con 0x7f4154102aa0 2024-04-25T14:59:01.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4151d9b700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4154103ca0 0x7f415419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:59:01.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f415259c700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f415419ae80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41440047b0 tx=0x7f4144009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:01.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f41437fe700 1 -- 172.21.15.138:0/2268768224 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144026070 con 0x7f4154102aa0 2024-04-25T14:59:01.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f41437fe700 1 -- 172.21.15.138:0/2268768224 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41440047e0 con 0x7f4154102aa0 2024-04-25T14:59:01.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f41437fe700 1 -- 172.21.15.138:0/2268768224 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144005320 con 0x7f4154102aa0 2024-04-25T14:59:01.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41541a2d70 con 0x7f4154102aa0 2024-04-25T14:59:01.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.349+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41541a31c0 con 0x7f4154102aa0 2024-04-25T14:59:01.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.350+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41541a3390 con 0x7f4154102aa0 2024-04-25T14:59:01.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.351+0000 7f41437fe700 1 -- 172.21.15.138:0/2268768224 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4144029070 con 0x7f4154102aa0 2024-04-25T14:59:01.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.352+0000 7f41437fe700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f413c0a3c40 0x7f413c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:01.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.352+0000 7f41437fe700 1 -- 172.21.15.138:0/2268768224 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f41440d0ab0 con 0x7f4154102aa0 2024-04-25T14:59:01.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.353+0000 7f4151d9b700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f413c0a3c40 0x7f413c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:01.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.354+0000 7f4151d9b700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f413c0a3c40 0x7f413c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4148007900 tx=0x7f4148008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:01.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.357+0000 7f41437fe700 1 -- 172.21.15.138:0/2268768224 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f414405dff0 con 0x7f4154102aa0 2024-04-25T14:59:01.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.568+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}) -- 0x7f415410ac90 con 0x7f413c0a3c40 2024-04-25T14:59:01.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.585+0000 7f41437fe700 1 -- 172.21.15.138:0/2268768224 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f415410ac90 con 0x7f413c0a3c40 2024-04-25T14:59:01.585 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mds.all update... 2024-04-25T14:59:01.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f413c0a3c40 msgr2=0x7f413c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:01.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f413c0a3c40 0x7f413c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4148007900 tx=0x7f4148008040 comp rx=0 tx=0).stop 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 msgr2=0x7f415419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f415419ae80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41440047b0 tx=0x7f4144009580 comp rx=0 tx=0).stop 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 shutdown_connections 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f413c0a3c40 0x7f413c0a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4154104640 0x7f41541a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4154102aa0 0x7f415419ae80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 --2- 172.21.15.138:0/2268768224 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4154103ca0 0x7f415419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.586+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 >> 172.21.15.138:0/2268768224 conn(0x7f41540fe040 msgr2=0x7f4154109470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.587+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 shutdown_connections 2024-04-25T14:59:01.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T14:59:01.587+0000 7f4158ad4700 1 -- 172.21.15.138:0/2268768224 wait complete. 2024-04-25T14:59:02.168 DEBUG:teuthology.orchestra.run.smithi094:mds.a> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T14:59:02.174 DEBUG:teuthology.orchestra.run.smithi094:mds.d> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T14:59:02.223 DEBUG:teuthology.orchestra.run.smithi094:mds.g> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T14:59:02.269 DEBUG:teuthology.orchestra.run.smithi094:mds.j> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T14:59:02.313 DEBUG:teuthology.orchestra.run.smithi137:mds.b> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T14:59:02.316 DEBUG:teuthology.orchestra.run.smithi137:mds.e> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T14:59:02.358 DEBUG:teuthology.orchestra.run.smithi137:mds.h> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T14:59:02.401 DEBUG:teuthology.orchestra.run.smithi137:mds.k> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T14:59:02.444 DEBUG:teuthology.orchestra.run.smithi138:mds.c> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T14:59:02.447 DEBUG:teuthology.orchestra.run.smithi138:mds.f> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T14:59:02.489 DEBUG:teuthology.orchestra.run.smithi138:mds.i> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T14:59:02.532 DEBUG:teuthology.orchestra.run.smithi138:mds.l> sudo journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T14:59:02.575 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T14:59:02.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:59:02.585 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:02 smithi094 ceph-mon[27862]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:02 smithi138 ceph-mon[32875]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:02 smithi138 ceph-mon[32875]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:02 smithi138 ceph-mon[32875]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T14:59:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:02 smithi137 ceph-mon[32439]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:02 smithi137 ceph-mon[32439]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:02 smithi137 ceph-mon[32439]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T14:59:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:02.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.814+0000 7fe8d0000640 1 -- 172.21.15.94:0/2693323260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 msgr2=0x7fe8c8109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:02.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.814+0000 7fe8d0000640 1 --2- 172.21.15.94:0/2693323260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 0x7fe8c8109c20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009900 tx=0x7fe8b402a410 comp rx=0 tx=0).stop 2024-04-25T14:59:02.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.814+0000 7fe8d0000640 1 -- 172.21.15.94:0/2693323260 shutdown_connections 2024-04-25T14:59:02.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.814+0000 7fe8d0000640 1 --2- 172.21.15.94:0/2693323260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c810a160 0x7fe8c81109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.814+0000 7fe8d0000640 1 --2- 172.21.15.94:0/2693323260 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 0x7fe8c8109c20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.814+0000 7fe8d0000640 1 --2- 172.21.15.94:0/2693323260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c81085b0 0x7fe8c81089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.814+0000 7fe8d0000640 1 -- 172.21.15.94:0/2693323260 >> 172.21.15.94:0/2693323260 conn(0x7fe8c8077570 msgr2=0x7fe8c8079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.815+0000 7fe8d0000640 1 -- 172.21.15.94:0/2693323260 shutdown_connections 2024-04-25T14:59:02.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.815+0000 7fe8d0000640 1 -- 172.21.15.94:0/2693323260 wait complete. 2024-04-25T14:59:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 Processor -- start 2024-04-25T14:59:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 -- start start 2024-04-25T14:59:02.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c81085b0 0x7fe8c819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 0x7fe8c819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c810a160 0x7fe8c81a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8114110 con 0x7fe8c810a160 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8113f90 con 0x7fe8c81097a0 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.816+0000 7fe8d0000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8114290 con 0x7fe8c81085b0 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c81085b0 0x7fe8c819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c81085b0 0x7fe8c819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:46764/0 (socket says 172.21.15.94:46764) 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 -- 172.21.15.94:0/148611747 learned_addr learned my addr 172.21.15.94:0/148611747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8ccc00640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 0x7fe8c819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8ce000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c810a160 0x7fe8c81a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 -- 172.21.15.94:0/148611747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 msgr2=0x7fe8c819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 0x7fe8c819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:02.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 -- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c810a160 msgr2=0x7fe8c81a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:02.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c810a160 0x7fe8c81a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:02.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 -- 172.21.15.94:0/148611747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c81a6410 con 0x7fe8c81085b0 2024-04-25T14:59:02.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8ce000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c810a160 0x7fe8c81a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:59:02.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.817+0000 7fe8cd600640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c81085b0 0x7fe8c819e7f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00e9c0 tx=0x7fe8bc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:02.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.818+0000 7fe8c6000640 1 -- 172.21.15.94:0/148611747 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc00cd80 con 0x7fe8c81085b0 2024-04-25T14:59:02.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.818+0000 7fe8c6000640 1 -- 172.21.15.94:0/148611747 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8bc0035f0 con 0x7fe8c81085b0 2024-04-25T14:59:02.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.818+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c81a6700 con 0x7fe8c81085b0 2024-04-25T14:59:02.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.818+0000 7fe8c6000640 1 -- 172.21.15.94:0/148611747 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc010690 con 0x7fe8c81085b0 2024-04-25T14:59:02.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.818+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81a6c20 con 0x7fe8c81085b0 2024-04-25T14:59:02.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.819+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c810f950 con 0x7fe8c81085b0 2024-04-25T14:59:02.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.821+0000 7fe8c6000640 1 -- 172.21.15.94:0/148611747 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe8bc0037a0 con 0x7fe8c81085b0 2024-04-25T14:59:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.823+0000 7fe8c6000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe8b00a3bb0 0x7fe8b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:02.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.823+0000 7fe8c6000640 1 -- 172.21.15.94:0/148611747 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8bc0d04d0 con 0x7fe8c81085b0 2024-04-25T14:59:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.823+0000 7fe8ccc00640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe8b00a3bb0 0x7fe8b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:02.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.824+0000 7fe8ccc00640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe8b00a3bb0 0x7fe8b00a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4000c00 tx=0x7fe8b40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:02.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:02.829+0000 7fe8c6000640 1 -- 172.21.15.94:0/148611747 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe8bc09d2b0 con 0x7fe8c81085b0 2024-04-25T14:59:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:02 smithi094 ceph-mon[27862]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:02 smithi094 ceph-mon[27862]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T14:59:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:03.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.033+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8c8062b50 con 0x7fe8c81085b0 2024-04-25T14:59:03.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.035+0000 7fe8c6000640 1 -- 172.21.15.94:0/148611747 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18441 (secure 0 0 0) 0x7fe8bc09cbb0 con 0x7fe8c81085b0 2024-04-25T14:59:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:03.039 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T14:58:52.443830+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe8b00a3bb0 msgr2=0x7fe8b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe8b00a3bb0 0x7fe8b00a6070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4000c00 tx=0x7fe8b40314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c81085b0 msgr2=0x7fe8c819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c81085b0 0x7fe8c819e7f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00e9c0 tx=0x7fe8bc00ee80 comp rx=0 tx=0).stop 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 shutdown_connections 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe8b00a3bb0 0x7fe8b00a6070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c810a160 0x7fe8c81a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe8c81097a0 0x7fe8c819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 --2- 172.21.15.94:0/148611747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8c81085b0 0x7fe8c819e7f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 >> 172.21.15.94:0/148611747 conn(0x7fe8c8077570 msgr2=0x7fe8c810cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 shutdown_connections 2024-04-25T14:59:03.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.038+0000 7fe8d0000640 1 -- 172.21.15.94:0/148611747 wait complete. 2024-04-25T14:59:03.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:59:03.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:59:03.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 -- 172.21.15.94:0/2172237910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330104090 msgr2=0x7f53301044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 --2- 172.21.15.94:0/2172237910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330104090 0x7f53301044b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5320009900 tx=0x7f532002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 -- 172.21.15.94:0/2172237910 shutdown_connections 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 --2- 172.21.15.94:0/2172237910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5330105c40 0x7f533010c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 --2- 172.21.15.94:0/2172237910 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 0x7f5330105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 --2- 172.21.15.94:0/2172237910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330104090 0x7f53301044b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 -- 172.21.15.94:0/2172237910 >> 172.21.15.94:0/2172237910 conn(0x7f53300ff600 msgr2=0x7f5330101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 -- 172.21.15.94:0/2172237910 shutdown_connections 2024-04-25T14:59:03.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.263+0000 7f532fe00640 1 -- 172.21.15.94:0/2172237910 wait complete. 2024-04-25T14:59:03.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.264+0000 7f532fe00640 1 Processor -- start 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.264+0000 7f532fe00640 1 -- start start 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5330104090 0x7f533019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 0x7f533019a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330105c40 0x7f53301a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532fe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f533010fbd0 con 0x7f5330104090 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532fe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f533010fa50 con 0x7f5330105280 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f533010fd50 con 0x7f5330105c40 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f5327e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 0x7f533019a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330105c40 0x7f53301a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f532ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5330104090 0x7f533019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f5327e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 0x7f533019a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42120/0 (socket says 172.21.15.94:42120) 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.265+0000 7f5327e00640 1 -- 172.21.15.94:0/3297967434 learned_addr learned my addr 172.21.15.94:0/3297967434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327e00640 1 -- 172.21.15.94:0/3297967434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330105c40 msgr2=0x7f53301a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327e00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330105c40 0x7f53301a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327e00640 1 -- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5330104090 msgr2=0x7f533019a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327e00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5330104090 0x7f533019a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327e00640 1 -- 172.21.15.94:0/3297967434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53301a1f10 con 0x7f5330105280 2024-04-25T14:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327e00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 0x7f533019a830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f531800ea60 tx=0x7f531800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327400640 1 -- 172.21.15.94:0/3297967434 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531800cde0 con 0x7f5330105280 2024-04-25T14:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327400640 1 -- 172.21.15.94:0/3297967434 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53180035f0 con 0x7f5330105280 2024-04-25T14:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f5327400640 1 -- 172.21.15.94:0/3297967434 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318010690 con 0x7f5330105280 2024-04-25T14:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53301a2200 con 0x7f5330105280 2024-04-25T14:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.266+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53301a2700 con 0x7f5330105280 2024-04-25T14:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.267+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f533010b450 con 0x7f5330105280 2024-04-25T14:59:03.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.269+0000 7f5327400640 1 -- 172.21.15.94:0/3297967434 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f53180037a0 con 0x7f5330105280 2024-04-25T14:59:03.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.270+0000 7f5327400640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f53140a3c30 0x7f53140a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.270+0000 7f5327400640 1 -- 172.21.15.94:0/3297967434 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f53180d0210 con 0x7f5330105280 2024-04-25T14:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.271+0000 7f532ea00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f53140a3c30 0x7f53140a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.271+0000 7f532ea00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f53140a3c30 0x7f53140a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5320009230 tx=0x7f53200314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:03.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.275+0000 7f5327400640 1 -- 172.21.15.94:0/3297967434 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f531809cff0 con 0x7f5330105280 2024-04-25T14:59:03.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.480+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5330062b50 con 0x7f5330105280 2024-04-25T14:59:03.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.481+0000 7f5327400640 1 -- 172.21.15.94:0/3297967434 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f531809c8f0 con 0x7f5330105280 2024-04-25T14:59:03.483 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f53140a3c30 msgr2=0x7f53140a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f53140a3c30 0x7f53140a60f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5320009230 tx=0x7f53200314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 msgr2=0x7f533019a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 0x7f533019a830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f531800ea60 tx=0x7f531800ef20 comp rx=0 tx=0).stop 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 shutdown_connections 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f53140a3c30 0x7f53140a60f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5330105c40 0x7f53301a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5330105280 0x7f533019a830 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 --2- 172.21.15.94:0/3297967434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5330104090 0x7f533019a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.484+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 >> 172.21.15.94:0/3297967434 conn(0x7f53300ff600 msgr2=0x7f5330108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:03.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.485+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 shutdown_connections 2024-04-25T14:59:03.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.485+0000 7f532fe00640 1 -- 172.21.15.94:0/3297967434 wait complete. 2024-04-25T14:59:03.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:03.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:03.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:03.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:03.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:03.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:03.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:03.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/148611747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:03.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:03 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3297967434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.706+0000 7f5db3000640 1 -- 172.21.15.94:0/366378827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac0784f0 msgr2=0x7f5dac078d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.706+0000 7f5db3000640 1 --2- 172.21.15.94:0/366378827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac0784f0 0x7f5dac078d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009860 tx=0x7f5d9c02a340 comp rx=0 tx=0).stop 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.707+0000 7f5db3000640 1 -- 172.21.15.94:0/366378827 shutdown_connections 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.707+0000 7f5db3000640 1 --2- 172.21.15.94:0/366378827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac0784f0 0x7f5dac078d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.707+0000 7f5db3000640 1 --2- 172.21.15.94:0/366378827 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac077b50 0x7f5dac077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.707+0000 7f5db3000640 1 --2- 172.21.15.94:0/366378827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dac076910 0x7f5dac076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.707+0000 7f5db3000640 1 -- 172.21.15.94:0/366378827 >> 172.21.15.94:0/366378827 conn(0x7f5dac0ff740 msgr2=0x7f5dac101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.707+0000 7f5db3000640 1 -- 172.21.15.94:0/366378827 shutdown_connections 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.707+0000 7f5db3000640 1 -- 172.21.15.94:0/366378827 wait complete. 2024-04-25T14:59:03.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 Processor -- start 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 -- start start 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac076910 0x7f5dac19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac077b50 0x7f5dac19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dac0784f0 0x7f5dac1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114110 con 0x7f5dac077b50 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac113f90 con 0x7f5dac076910 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.708+0000 7f5db3000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114290 con 0x7f5dac0784f0 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac077b50 0x7f5dac19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac077b50 0x7f5dac19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39086/0 (socket says 172.21.15.94:39086) 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 -- 172.21.15.94:0/1809784820 learned_addr learned my addr 172.21.15.94:0/1809784820 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dabe00640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac076910 0x7f5dac19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5db1000640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dac0784f0 0x7f5dac1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 -- 172.21.15.94:0/1809784820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dac0784f0 msgr2=0x7f5dac1a5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dac0784f0 0x7f5dac1a5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 -- 172.21.15.94:0/1809784820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac076910 msgr2=0x7f5dac19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac076910 0x7f5dac19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 -- 172.21.15.94:0/1809784820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac1a63a0 con 0x7f5dac077b50 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.709+0000 7f5dab400640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac077b50 0x7f5dac19ecc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.710+0000 7f5da8c00640 1 -- 172.21.15.94:0/1809784820 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400cd80 con 0x7f5dac077b50 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.710+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac1a6690 con 0x7f5dac077b50 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.710+0000 7f5da8c00640 1 -- 172.21.15.94:0/1809784820 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940108e0 con 0x7f5dac077b50 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.710+0000 7f5da8c00640 1 -- 172.21.15.94:0/1809784820 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94018980 con 0x7f5dac077b50 2024-04-25T14:59:03.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.710+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac1a6be0 con 0x7f5dac077b50 2024-04-25T14:59:03.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.711+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dac110d20 con 0x7f5dac077b50 2024-04-25T14:59:03.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.713+0000 7f5da8c00640 1 -- 172.21.15.94:0/1809784820 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5d94010430 con 0x7f5dac077b50 2024-04-25T14:59:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.716+0000 7f5da8c00640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d900a3bb0 0x7f5d900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.717+0000 7f5da8c00640 1 -- 172.21.15.94:0/1809784820 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5d940d02d0 con 0x7f5dac077b50 2024-04-25T14:59:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.717+0000 7f5dabe00640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d900a3bb0 0x7f5d900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.717+0000 7f5dabe00640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d900a3bb0 0x7f5d900a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5da4005fd0 tx=0x7f5da4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:03.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.722+0000 7f5da8c00640 1 -- 172.21.15.94:0/1809784820 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d9409ca60 con 0x7f5dac077b50 2024-04-25T14:59:03.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:03.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:03.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:03.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:03.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:03.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:03.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/148611747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:03.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:03 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3297967434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:59:03.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.967+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dac062b50 con 0x7f5dac077b50 2024-04-25T14:59:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.968+0000 7f5da8c00640 1 -- 172.21.15.94:0/1809784820 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f5d940a0d60 con 0x7f5dac077b50 2024-04-25T14:59:03.969 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:03.969 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T14:59:03.969 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 1 2024-04-25T14:59:03.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d900a3bb0 msgr2=0x7f5d900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d900a3bb0 0x7f5d900a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5da4005fd0 tx=0x7f5da4010040 comp rx=0 tx=0).stop 2024-04-25T14:59:03.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac077b50 msgr2=0x7f5dac19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:03.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac077b50 0x7f5dac19ecc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 shutdown_connections 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d900a3bb0 0x7f5d900a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dac0784f0 0x7f5dac1a5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dac077b50 0x7f5dac19ecc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 --2- 172.21.15.94:0/1809784820 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5dac076910 0x7f5dac19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 >> 172.21.15.94:0/1809784820 conn(0x7f5dac0ff740 msgr2=0x7f5dac100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 shutdown_connections 2024-04-25T14:59:03.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:03.971+0000 7f5db3000640 1 -- 172.21.15.94:0/1809784820 wait complete. 2024-04-25T14:59:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:03.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/148611747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:03.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:03 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3297967434' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:59:04.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:04 smithi138 ceph-mon[32875]: Deploying daemon mds.l on smithi138 2024-04-25T14:59:04.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:04 smithi138 ceph-mon[32875]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:04.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:04 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1809784820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:04.958 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:04 smithi138 systemd[1]: Starting Ceph mds.l for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:04 smithi094 ceph-mon[27862]: Deploying daemon mds.l on smithi138 2024-04-25T14:59:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:04 smithi094 ceph-mon[27862]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:04 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1809784820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:04 smithi137 ceph-mon[32439]: Deploying daemon mds.l on smithi138 2024-04-25T14:59:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:04 smithi137 ceph-mon[32439]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:04 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1809784820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:05.279 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 podman[63078]: 2024-04-25 14:59:05.009340443 +0000 UTC m=+0.110568392 container create e7ad3ca004a585f30b11b1a2e6a74c7d462397785f4152169d546e569444d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:59:05.279 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 podman[63078]: 2024-04-25 14:59:04.916668298 +0000 UTC m=+0.017896221 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:05.279 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 podman[63078]: 2024-04-25 14:59:05.195180102 +0000 UTC m=+0.296408009 container init e7ad3ca004a585f30b11b1a2e6a74c7d462397785f4152169d546e569444d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T14:59:05.279 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 podman[63078]: 2024-04-25 14:59:05.197733004 +0000 UTC m=+0.298960919 container start e7ad3ca004a585f30b11b1a2e6a74c7d462397785f4152169d546e569444d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T14:59:05.279 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 ceph-mds[63097]: -- 172.21.15.138:0/767558006 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556901441a40 con 0x556901415000 2024-04-25T14:59:05.279 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l[63093]: starting mds.l at 2024-04-25T14:59:05.279 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 bash[63078]: e7ad3ca004a585f30b11b1a2e6a74c7d462397785f4152169d546e569444d54d 2024-04-25T14:59:05.707 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 14:59:05 smithi138 systemd[1]: Started Ceph mds.l for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:05.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 -- 172.21.15.94:0/2268224667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 msgr2=0x7ff61c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 --2- 172.21.15.94:0/2268224667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c1087f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff610009860 tx=0x7ff61002a340 comp rx=0 tx=0).stop 2024-04-25T14:59:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 -- 172.21.15.94:0/2268224667 shutdown_connections 2024-04-25T14:59:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 --2- 172.21.15.94:0/2268224667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff61c109f80 0x7ff61c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 --2- 172.21.15.94:0/2268224667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff61c1095c0 0x7ff61c109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 --2- 172.21.15.94:0/2268224667 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c1087f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 -- 172.21.15.94:0/2268224667 >> 172.21.15.94:0/2268224667 conn(0x7ff61c077390 msgr2=0x7ff61c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 -- 172.21.15.94:0/2268224667 shutdown_connections 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.204+0000 7ff622c00640 1 -- 172.21.15.94:0/2268224667 wait complete. 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.205+0000 7ff622c00640 1 Processor -- start 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622c00640 1 -- start start 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff61c1095c0 0x7ff61c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff61c109f80 0x7ff61c1a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c113ee0 con 0x7ff61c109f80 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c113d60 con 0x7ff61c1083d0 2024-04-25T14:59:06.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c114060 con 0x7ff61c1095c0 2024-04-25T14:59:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff621800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff621800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:42166/0 (socket says 172.21.15.94:42166) 2024-04-25T14:59:06.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff621800640 1 -- 172.21.15.94:0/1687116884 learned_addr learned my addr 172.21.15.94:0/1687116884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff622200640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff61c109f80 0x7ff61c1a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.206+0000 7ff620e00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff61c1095c0 0x7ff61c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff621800640 1 -- 172.21.15.94:0/1687116884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff61c1095c0 msgr2=0x7ff61c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff621800640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff61c1095c0 0x7ff61c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff621800640 1 -- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff61c109f80 msgr2=0x7ff61c1a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff621800640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff61c109f80 0x7ff61c1a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff621800640 1 -- 172.21.15.94:0/1687116884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c1a6280 con 0x7ff61c1083d0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff620e00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff61c1095c0 0x7ff61c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff621800640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c19e660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff6100057c0 tx=0x7ff610032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff61a000640 1 -- 172.21.15.94:0/1687116884 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610037040 con 0x7ff61c1083d0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff61a000640 1 -- 172.21.15.94:0/1687116884 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff610030bd0 con 0x7ff61c1083d0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff61a000640 1 -- 172.21.15.94:0/1687116884 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610005280 con 0x7ff61c1083d0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c1a6510 con 0x7ff61c1083d0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.207+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c1a6950 con 0x7ff61c1083d0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.208+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c10f810 con 0x7ff61c1083d0 2024-04-25T14:59:06.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.210+0000 7ff61a000640 1 -- 172.21.15.94:0/1687116884 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff610043020 con 0x7ff61c1083d0 2024-04-25T14:59:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.212+0000 7ff61a000640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6040a3c30 0x7ff6040a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.212+0000 7ff61a000640 1 -- 172.21.15.94:0/1687116884 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff6100ef630 con 0x7ff61c1083d0 2024-04-25T14:59:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.212+0000 7ff620e00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6040a3c30 0x7ff6040a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:06.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.213+0000 7ff620e00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6040a3c30 0x7ff6040a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff61c19fb70 tx=0x7ff608005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:06.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.216+0000 7ff61a000640 1 -- 172.21.15.94:0/1687116884 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6100bc410 con 0x7ff61c1083d0 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.138:6832/2043270325,v1:172.21.15.138:6833/2043270325] up:boot 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: fsmap 1 up:standby 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: Deploying daemon mds.k on smithi137 2024-04-25T14:59:06.319 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:06 smithi137 ceph-mon[32439]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.138:6832/2043270325,v1:172.21.15.138:6833/2043270325] up:boot 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: fsmap 1 up:standby 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:06.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:06.464 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: Deploying daemon mds.k on smithi137 2024-04-25T14:59:06.464 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:06 smithi094 ceph-mon[27862]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:06.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.464+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff61c02d5c0 con 0x7ff61c1083d0 2024-04-25T14:59:06.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.465+0000 7ff61a000640 1 -- 172.21.15.94:0/1687116884 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7ff6100bbd10 con 0x7ff61c1083d0 2024-04-25T14:59:06.470 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:06.470 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T14:59:06.470 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-04-25T14:59:06.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6040a3c30 msgr2=0x7ff6040a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6040a3c30 0x7ff6040a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff61c19fb70 tx=0x7ff608005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 msgr2=0x7ff61c19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c19e660 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff6100057c0 tx=0x7ff610032da0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 shutdown_connections 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6040a3c30 0x7ff6040a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff61c109f80 0x7ff61c1a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff61c1095c0 0x7ff61c19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 --2- 172.21.15.94:0/1687116884 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff61c1083d0 0x7ff61c19e660 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 >> 172.21.15.94:0/1687116884 conn(0x7ff61c077390 msgr2=0x7ff61c10c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 shutdown_connections 2024-04-25T14:59:06.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:06.468+0000 7ff622c00640 1 -- 172.21.15.94:0/1687116884 wait complete. 2024-04-25T14:59:06.608 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.138:6832/2043270325,v1:172.21.15.138:6833/2043270325] up:boot 2024-04-25T14:59:06.608 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: fsmap 1 up:standby 2024-04-25T14:59:06.608 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T14:59:06.608 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.608 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.609 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:06.609 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:06.609 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:06.609 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:06.609 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: Deploying daemon mds.k on smithi137 2024-04-25T14:59:06.609 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:06 smithi138 ceph-mon[32875]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:07.647 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:07 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1687116884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:07.647 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:07 smithi137 systemd[1]: Starting Ceph mds.k for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:07 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1687116884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:07 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1687116884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:07.949 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:07 smithi137 podman[62327]: 2024-04-25 14:59:07.646253822 +0000 UTC m=+0.100424622 container create a93f09d911cc5bfd68fba86051a513547bd85bae4ac2220656c356fd5cf16441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T14:59:07.949 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:07 smithi137 podman[62327]: 2024-04-25 14:59:07.563705274 +0000 UTC m=+0.017876084 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:07.949 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:07 smithi137 podman[62327]: 2024-04-25 14:59:07.949309928 +0000 UTC m=+0.403480729 container init a93f09d911cc5bfd68fba86051a513547bd85bae4ac2220656c356fd5cf16441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T14:59:08.232 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:07 smithi137 podman[62327]: 2024-04-25 14:59:07.951917462 +0000 UTC m=+0.406088264 container start a93f09d911cc5bfd68fba86051a513547bd85bae4ac2220656c356fd5cf16441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:59:08.233 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:07 smithi137 ceph-mds[62346]: -- 172.21.15.137:0/2944976617 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559dc2599a40 con 0x559dc256c800 2024-04-25T14:59:08.233 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k[62342]: starting mds.k at 2024-04-25T14:59:08.233 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:08 smithi137 bash[62327]: a93f09d911cc5bfd68fba86051a513547bd85bae4ac2220656c356fd5cf16441 2024-04-25T14:59:08.233 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 14:59:08 smithi137 systemd[1]: Started Ceph mds.k for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:08.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 -- 172.21.15.94:0/2284165802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4101ec0 msgr2=0x7f2ad4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/2284165802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4101ec0 0x7f2ad4102340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009900 tx=0x7f2abc02a410 comp rx=0 tx=0).stop 2024-04-25T14:59:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 -- 172.21.15.94:0/2284165802 shutdown_connections 2024-04-25T14:59:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/2284165802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad4102880 0x7f2ad4109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/2284165802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4101ec0 0x7f2ad4102340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/2284165802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ad4100cd0 0x7f2ad41010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 -- 172.21.15.94:0/2284165802 >> 172.21.15.94:0/2284165802 conn(0x7f2ad40fc220 msgr2=0x7f2ad40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 -- 172.21.15.94:0/2284165802 shutdown_connections 2024-04-25T14:59:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.694+0000 7f2ad8e00640 1 -- 172.21.15.94:0/2284165802 wait complete. 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.695+0000 7f2ad8e00640 1 Processor -- start 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.695+0000 7f2ad8e00640 1 -- start start 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad8e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4100cd0 0x7f2ad4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad4101ec0 0x7f2ad4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad8e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ad4102880 0x7f2ad419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad8e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad410c7e0 con 0x7f2ad4100cd0 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad8e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad410c660 con 0x7f2ad4102880 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad8e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad410c960 con 0x7f2ad4101ec0 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4100cd0 0x7f2ad4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad3400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4100cd0 0x7f2ad4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45320/0 (socket says 172.21.15.94:45320) 2024-04-25T14:59:08.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad3400640 1 -- 172.21.15.94:0/929874678 learned_addr learned my addr 172.21.15.94:0/929874678 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2ad3e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ad4102880 0x7f2ad419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.696+0000 7f2acbe00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad4101ec0 0x7f2ad4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad3400640 1 -- 172.21.15.94:0/929874678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad4101ec0 msgr2=0x7f2ad4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad3400640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad4101ec0 0x7f2ad4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad3400640 1 -- 172.21.15.94:0/929874678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ad4102880 msgr2=0x7f2ad419f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad3400640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ad4102880 0x7f2ad419f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad3400640 1 -- 172.21.15.94:0/929874678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad419fc30 con 0x7f2ad4100cd0 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad3e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ad4102880 0x7f2ad419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad3400640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4100cd0 0x7f2ad4198010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400e9c0 tx=0x7f2ac400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad0c00640 1 -- 172.21.15.94:0/929874678 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac400cd80 con 0x7f2ad4100cd0 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad0c00640 1 -- 172.21.15.94:0/929874678 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ac40035f0 con 0x7f2ad4100cd0 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad419ff20 con 0x7f2ad4100cd0 2024-04-25T14:59:08.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad0c00640 1 -- 172.21.15.94:0/929874678 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4010690 con 0x7f2ad4100cd0 2024-04-25T14:59:08.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.697+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad41a0380 con 0x7f2ad4100cd0 2024-04-25T14:59:08.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.698+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad41a0610 con 0x7f2ad4100cd0 2024-04-25T14:59:08.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.702+0000 7f2ad0c00640 1 -- 172.21.15.94:0/929874678 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2ac40037a0 con 0x7f2ad4100cd0 2024-04-25T14:59:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.703+0000 7f2ad0c00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2ab80a3bb0 0x7f2ab80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.704+0000 7f2ad0c00640 1 -- 172.21.15.94:0/929874678 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2ac40d0370 con 0x7f2ad4100cd0 2024-04-25T14:59:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.704+0000 7f2acbe00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2ab80a3bb0 0x7f2ab80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:08.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.705+0000 7f2acbe00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2ab80a3bb0 0x7f2ab80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2abc000c00 tx=0x7f2abc0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:08 smithi138 ceph-mon[32875]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:08.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:08 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:08.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.706+0000 7f2ad0c00640 1 -- 172.21.15.94:0/929874678 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2ac409d1f0 con 0x7f2ad4100cd0 2024-04-25T14:59:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:08 smithi094 ceph-mon[27862]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:08.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:08 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:08.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:08 smithi137 ceph-mon[32439]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:08.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:08.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:08.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:08.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:08 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:08.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.961+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ad41a09a0 con 0x7f2ad4100cd0 2024-04-25T14:59:08.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.962+0000 7f2ad0c00640 1 -- 172.21.15.94:0/929874678 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f2ac409caf0 con 0x7f2ad4100cd0 2024-04-25T14:59:08.964 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:08.964 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T14:59:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-04-25T14:59:08.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2ab80a3bb0 msgr2=0x7f2ab80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:08.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2ab80a3bb0 0x7f2ab80a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2abc000c00 tx=0x7f2abc0314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4100cd0 msgr2=0x7f2ad4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:08.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4100cd0 0x7f2ad4198010 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400e9c0 tx=0x7f2ac400ee80 comp rx=0 tx=0).stop 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 shutdown_connections 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2ab80a3bb0 0x7f2ab80a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2ad4102880 0x7f2ad419f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ad4101ec0 0x7f2ad4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 --2- 172.21.15.94:0/929874678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ad4100cd0 0x7f2ad4198010 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 >> 172.21.15.94:0/929874678 conn(0x7f2ad40fc220 msgr2=0x7f2ad40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 shutdown_connections 2024-04-25T14:59:08.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:08.965+0000 7f2ad8e00640 1 -- 172.21.15.94:0/929874678 wait complete. 2024-04-25T14:59:09.563 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:09 smithi094 ceph-mon[27862]: Deploying daemon mds.j on smithi094 2024-04-25T14:59:09.563 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:09 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.137:6832/2696996187,v1:172.21.15.137:6833/2696996187] up:boot 2024-04-25T14:59:09.563 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:09 smithi094 ceph-mon[27862]: fsmap 2 up:standby 2024-04-25T14:59:09.564 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:09 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:59:09.564 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:09 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/929874678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:09 smithi138 ceph-mon[32875]: Deploying daemon mds.j on smithi094 2024-04-25T14:59:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:09 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.137:6832/2696996187,v1:172.21.15.137:6833/2696996187] up:boot 2024-04-25T14:59:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:09 smithi138 ceph-mon[32875]: fsmap 2 up:standby 2024-04-25T14:59:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:09 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:59:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:09 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/929874678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:09 smithi137 ceph-mon[32439]: Deploying daemon mds.j on smithi094 2024-04-25T14:59:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:09 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.137:6832/2696996187,v1:172.21.15.137:6833/2696996187] up:boot 2024-04-25T14:59:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:09 smithi137 ceph-mon[32439]: fsmap 2 up:standby 2024-04-25T14:59:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:09 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T14:59:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:09 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/929874678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:10.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:10 smithi138 ceph-mon[32875]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:10.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:10 smithi094 ceph-mon[27862]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:10.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:10 smithi137 ceph-mon[32439]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:10.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 -- 172.21.15.94:0/3208004013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c105d80 msgr2=0x7f7f1c10c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 --2- 172.21.15.94:0/3208004013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c105d80 0x7f7f1c10c610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009ca0 tx=0x7f7f1402a8b0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 -- 172.21.15.94:0/3208004013 shutdown_connections 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 --2- 172.21.15.94:0/3208004013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c105d80 0x7f7f1c10c610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 --2- 172.21.15.94:0/3208004013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1053c0 0x7f7f1c105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 --2- 172.21.15.94:0/3208004013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f1c1041d0 0x7f7f1c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 -- 172.21.15.94:0/3208004013 >> 172.21.15.94:0/3208004013 conn(0x7f7f1c0ff740 msgr2=0x7f7f1c101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 -- 172.21.15.94:0/3208004013 shutdown_connections 2024-04-25T14:59:11.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.192+0000 7f7f22a00640 1 -- 172.21.15.94:0/3208004013 wait complete. 2024-04-25T14:59:11.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.194+0000 7f7f22a00640 1 Processor -- start 2024-04-25T14:59:11.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.194+0000 7f7f22a00640 1 -- start start 2024-04-25T14:59:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.194+0000 7f7f22a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1041d0 0x7f7f1c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.194+0000 7f7f22a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c1053c0 0x7f7f1c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f22a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f1c105d80 0x7f7f1c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f22a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10fd30 con 0x7f7f1c1041d0 2024-04-25T14:59:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f22a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10fbb0 con 0x7f7f1c1053c0 2024-04-25T14:59:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f22a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10feb0 con 0x7f7f1c105d80 2024-04-25T14:59:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1041d0 0x7f7f1c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:11.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1041d0 0x7f7f1c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45334/0 (socket says 172.21.15.94:45334) 2024-04-25T14:59:11.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 -- 172.21.15.94:0/1195262152 learned_addr learned my addr 172.21.15.94:0/1195262152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:11.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1b400640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c1053c0 0x7f7f1c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:11.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f20a00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f1c105d80 0x7f7f1c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 -- 172.21.15.94:0/1195262152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f1c105d80 msgr2=0x7f7f1c1a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f1c105d80 0x7f7f1c1a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 -- 172.21.15.94:0/1195262152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c1053c0 msgr2=0x7f7f1c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c1053c0 0x7f7f1c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1be00640 1 -- 172.21.15.94:0/1195262152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c1a2000 con 0x7f7f1c1041d0 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.195+0000 7f7f1b400640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c1053c0 0x7f7f1c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.196+0000 7f7f1be00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1041d0 0x7f7f1c19a3e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9c0 tx=0x7f7f0c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.196+0000 7f7f18c00640 1 -- 172.21.15.94:0/1195262152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00cde0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.196+0000 7f7f18c00640 1 -- 172.21.15.94:0/1195262152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c0108e0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.196+0000 7f7f18c00640 1 -- 172.21.15.94:0/1195262152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0188d0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.196+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c1a22f0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.196+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a27f0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.198+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c10b4f0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.199+0000 7f7f18c00640 1 -- 172.21.15.94:0/1195262152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7f0c01f020 con 0x7f7f1c1041d0 2024-04-25T14:59:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.201+0000 7f7f18c00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7f000a3c30 0x7f7f000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.201+0000 7f7f18c00640 1 -- 172.21.15.94:0/1195262152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7f0c0d00f0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.201+0000 7f7f1b400640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7f000a3c30 0x7f7f000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:11.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.202+0000 7f7f1b400640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7f000a3c30 0x7f7f000a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c19b8f0 tx=0x7f7f0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:11.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.206+0000 7f7f18c00640 1 -- 172.21.15.94:0/1195262152 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f0c09ced0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.455 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 systemd[1]: Starting Ceph mds.j for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:11.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.458+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f1c02d5c0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.459+0000 7f7f18c00640 1 -- 172.21.15.94:0/1195262152 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7f7f0c09c7d0 con 0x7f7f1c1041d0 2024-04-25T14:59:11.460 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:11.460 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T14:59:11.460 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-04-25T14:59:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.461+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7f000a3c30 msgr2=0x7f7f000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.461+0000 7f7f22a00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7f000a3c30 0x7f7f000a60f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c19b8f0 tx=0x7f7f0800b410 comp rx=0 tx=0).stop 2024-04-25T14:59:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.461+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1041d0 msgr2=0x7f7f1c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.461+0000 7f7f22a00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1041d0 0x7f7f1c19a3e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9c0 tx=0x7f7f0c00ee80 comp rx=0 tx=0).stop 2024-04-25T14:59:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 shutdown_connections 2024-04-25T14:59:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7f000a3c30 0x7f7f000a60f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f1c105d80 0x7f7f1c1a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f1c1053c0 0x7f7f1c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 --2- 172.21.15.94:0/1195262152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f1c1041d0 0x7f7f1c19a3e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:11.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 >> 172.21.15.94:0/1195262152 conn(0x7f7f1c0ff740 msgr2=0x7f7f1c1086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:11.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 shutdown_connections 2024-04-25T14:59:11.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:11.462+0000 7f7f22a00640 1 -- 172.21.15.94:0/1195262152 wait complete. 2024-04-25T14:59:11.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:11 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1195262152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:11.713 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 podman[68184]: 2024-04-25 14:59:11.361719406 +0000 UTC m=+0.039260188 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:11.713 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 podman[68184]: 2024-04-25 14:59:11.471486629 +0000 UTC m=+0.149027414 container create 7579c7a12259c7a1ec95f869a3867c97be748644aa29d84bea0b94da47fcb88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T14:59:11.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:11 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1195262152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:11 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1195262152' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:12.213 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 podman[68184]: 2024-04-25 14:59:11.722588088 +0000 UTC m=+0.400128914 container init 7579c7a12259c7a1ec95f869a3867c97be748644aa29d84bea0b94da47fcb88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T14:59:12.213 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 podman[68184]: 2024-04-25 14:59:11.733307639 +0000 UTC m=+0.410848439 container start 7579c7a12259c7a1ec95f869a3867c97be748644aa29d84bea0b94da47fcb88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T14:59:12.213 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 ceph-mds[68205]: -- 172.21.15.94:0/3657255449 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ee7907e4e0 con 0x55ee79054c00 2024-04-25T14:59:12.213 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 ceph-mds[68205]: -- 172.21.15.94:0/3657255449 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee79081a40 con 0x55ee79054c00 2024-04-25T14:59:12.213 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j[68201]: starting mds.j at 2024-04-25T14:59:12.213 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 bash[68184]: 7579c7a12259c7a1ec95f869a3867c97be748644aa29d84bea0b94da47fcb88f 2024-04-25T14:59:12.213 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 14:59:11 smithi094 systemd[1]: Started Ceph mds.j for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:12.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:12 smithi094 ceph-mon[27862]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:12.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:12.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:12.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:12 smithi138 ceph-mon[32875]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:12 smithi137 ceph-mon[32439]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:13.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 -- 172.21.15.94:0/3664255082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1041b0 msgr2=0x7f521c1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 --2- 172.21.15.94:0/3664255082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1041b0 0x7f521c1045d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5214009ca0 tx=0x7f521402a8c0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 -- 172.21.15.94:0/3664255082 shutdown_connections 2024-04-25T14:59:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 --2- 172.21.15.94:0/3664255082 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f521c105d60 0x7f521c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 --2- 172.21.15.94:0/3664255082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f521c1053a0 0x7f521c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 --2- 172.21.15.94:0/3664255082 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1041b0 0x7f521c1045d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 -- 172.21.15.94:0/3664255082 >> 172.21.15.94:0/3664255082 conn(0x7f521c0ff740 msgr2=0x7f521c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:13.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 -- 172.21.15.94:0/3664255082 shutdown_connections 2024-04-25T14:59:13.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.694+0000 7f5221800640 1 -- 172.21.15.94:0/3664255082 wait complete. 2024-04-25T14:59:13.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.695+0000 7f5221800640 1 Processor -- start 2024-04-25T14:59:13.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.695+0000 7f5221800640 1 -- start start 2024-04-25T14:59:13.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f5221800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f521c1041b0 0x7f521c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:13.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f5221800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1053a0 0x7f521c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f5221800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f521c105d60 0x7f521c1a1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f5221800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f521c10fd40 con 0x7f521c1053a0 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f5221800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f521c10fbc0 con 0x7f521c1041b0 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f5221800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f521c10fec0 con 0x7f521c105d60 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1053a0 0x7f521c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1053a0 0x7f521c19a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45360/0 (socket says 172.21.15.94:45360) 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 -- 172.21.15.94:0/656627418 learned_addr learned my addr 172.21.15.94:0/656627418 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521aa00640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f521c1041b0 0x7f521c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521b400640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f521c105d60 0x7f521c1a1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 -- 172.21.15.94:0/656627418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f521c105d60 msgr2=0x7f521c1a1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f521c105d60 0x7f521c1a1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 -- 172.21.15.94:0/656627418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f521c1041b0 msgr2=0x7f521c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f521c1041b0 0x7f521c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.696+0000 7f521a000640 1 -- 172.21.15.94:0/656627418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521c1a2070 con 0x7f521c1053a0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.697+0000 7f521a000640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1053a0 0x7f521c19a990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f520400e9f0 tx=0x7f520400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.697+0000 7f5213400640 1 -- 172.21.15.94:0/656627418 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520400ccc0 con 0x7f521c1053a0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.697+0000 7f5213400640 1 -- 172.21.15.94:0/656627418 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f520400ce60 con 0x7f521c1053a0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.697+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521c1a2360 con 0x7f521c1053a0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.697+0000 7f5213400640 1 -- 172.21.15.94:0/656627418 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204010430 con 0x7f521c1053a0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.697+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521c1a2860 con 0x7f521c1053a0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.698+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521c10b540 con 0x7f521c1053a0 2024-04-25T14:59:13.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.700+0000 7f5213400640 1 -- 172.21.15.94:0/656627418 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f52040046c0 con 0x7f521c1053a0 2024-04-25T14:59:13.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.701+0000 7f5213400640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52000a3bb0 0x7f52000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:13.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.702+0000 7f5213400640 1 -- 172.21.15.94:0/656627418 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f52040d0330 con 0x7f521c1053a0 2024-04-25T14:59:13.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.702+0000 7f521aa00640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52000a3bb0 0x7f52000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:13.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.702+0000 7f521aa00640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52000a3bb0 0x7f52000a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5214000f80 tx=0x7f5214005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:13.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.706+0000 7f5213400640 1 -- 172.21.15.94:0/656627418 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f520409d1b0 con 0x7f521c1053a0 2024-04-25T14:59:13.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:13 smithi094 ceph-mon[27862]: Deploying daemon mds.d on smithi094 2024-04-25T14:59:13.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:13 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.94:6834/3392009193,v1:172.21.15.94:6835/3392009193] up:boot 2024-04-25T14:59:13.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:13 smithi094 ceph-mon[27862]: fsmap 3 up:standby 2024-04-25T14:59:13.845 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:59:13.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.953+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f521c02d5c0 con 0x7f521c1053a0 2024-04-25T14:59:13.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.954+0000 7f5213400640 1 -- 172.21.15.94:0/656627418 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f520409cab0 con 0x7f521c1053a0 2024-04-25T14:59:13.956 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:13.956 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T14:59:13.956 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 4 2024-04-25T14:59:13.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:13 smithi138 ceph-mon[32875]: Deploying daemon mds.d on smithi094 2024-04-25T14:59:13.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:13 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.94:6834/3392009193,v1:172.21.15.94:6835/3392009193] up:boot 2024-04-25T14:59:13.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:13 smithi138 ceph-mon[32875]: fsmap 3 up:standby 2024-04-25T14:59:13.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:59:13.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52000a3bb0 msgr2=0x7f52000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:13.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52000a3bb0 0x7f52000a6070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5214000f80 tx=0x7f5214005f30 comp rx=0 tx=0).stop 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1053a0 msgr2=0x7f521c19a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1053a0 0x7f521c19a990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f520400e9f0 tx=0x7f520400edf0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 shutdown_connections 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f52000a3bb0 0x7f52000a6070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f521c105d60 0x7f521c1a1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f521c1053a0 0x7f521c19a990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 --2- 172.21.15.94:0/656627418 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f521c1041b0 0x7f521c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 >> 172.21.15.94:0/656627418 conn(0x7f521c0ff740 msgr2=0x7f521c108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 shutdown_connections 2024-04-25T14:59:13.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:13.957+0000 7f5221800640 1 -- 172.21.15.94:0/656627418 wait complete. 2024-04-25T14:59:13.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:13 smithi137 ceph-mon[32439]: Deploying daemon mds.d on smithi094 2024-04-25T14:59:13.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:13 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.94:6834/3392009193,v1:172.21.15.94:6835/3392009193] up:boot 2024-04-25T14:59:13.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:13 smithi137 ceph-mon[32439]: fsmap 3 up:standby 2024-04-25T14:59:13.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T14:59:14.887 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:14 smithi094 ceph-mon[27862]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:14.887 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:14 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/656627418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:14 smithi138 ceph-mon[32875]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:14 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/656627418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:14 smithi137 ceph-mon[32439]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:14 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/656627418' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:15.213 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:14 smithi094 systemd[1]: Starting Ceph mds.d for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:15.532 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 podman[68533]: 2024-04-25 14:59:15.240153731 +0000 UTC m=+0.123793448 container create c99bf4f29c5a6dcf8dda0474442a8525f51d1294d95eaa0263dcaf7d6e090509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T14:59:15.532 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 podman[68533]: 2024-04-25 14:59:15.152106599 +0000 UTC m=+0.035746284 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:15.532 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 podman[68533]: 2024-04-25 14:59:15.530851729 +0000 UTC m=+0.414491470 container init c99bf4f29c5a6dcf8dda0474442a8525f51d1294d95eaa0263dcaf7d6e090509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5) 2024-04-25T14:59:15.962 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 podman[68533]: 2024-04-25 14:59:15.542776307 +0000 UTC m=+0.426416006 container start c99bf4f29c5a6dcf8dda0474442a8525f51d1294d95eaa0263dcaf7d6e090509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T14:59:15.963 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 ceph-mds[68551]: -- 172.21.15.94:0/2483243757 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d620f3a40 con 0x562d620c7000 2024-04-25T14:59:15.963 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d[68547]: starting mds.d at 2024-04-25T14:59:15.963 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 bash[68533]: c99bf4f29c5a6dcf8dda0474442a8525f51d1294d95eaa0263dcaf7d6e090509 2024-04-25T14:59:15.963 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 14:59:15 smithi094 systemd[1]: Started Ceph mds.d for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:15.977 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.190+0000 7fd4ed400640 1 -- 172.21.15.94:0/3319681230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 msgr2=0x7fd4e8105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.190+0000 7fd4ed400640 1 --2- 172.21.15.94:0/3319681230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 0x7fd4e8105640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009900 tx=0x7fd4d002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.190+0000 7fd4ed400640 1 -- 172.21.15.94:0/3319681230 shutdown_connections 2024-04-25T14:59:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.190+0000 7fd4ed400640 1 --2- 172.21.15.94:0/3319681230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8105b80 0x7fd4e810c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.190+0000 7fd4ed400640 1 --2- 172.21.15.94:0/3319681230 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 0x7fd4e8105640 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.190+0000 7fd4ed400640 1 --2- 172.21.15.94:0/3319681230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4e8103fd0 0x7fd4e81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.190+0000 7fd4ed400640 1 -- 172.21.15.94:0/3319681230 >> 172.21.15.94:0/3319681230 conn(0x7fd4e80ff560 msgr2=0x7fd4e8101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:16.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.191+0000 7fd4ed400640 1 -- 172.21.15.94:0/3319681230 shutdown_connections 2024-04-25T14:59:16.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.191+0000 7fd4ed400640 1 -- 172.21.15.94:0/3319681230 wait complete. 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 Processor -- start 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 -- start start 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8103fd0 0x7fd4e819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 0x7fd4e819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4e8105b80 0x7fd4e81a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e810fb10 con 0x7fd4e8103fd0 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e810f990 con 0x7fd4e81051c0 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.192+0000 7fd4ed400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e810fc90 con 0x7fd4e8105b80 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8103fd0 0x7fd4e819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8103fd0 0x7fd4e819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45402/0 (socket says 172.21.15.94:45402) 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 -- 172.21.15.94:0/1716974412 learned_addr learned my addr 172.21.15.94:0/1716974412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:16.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4df400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 0x7fd4e819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:16.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4eca00640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4e8105b80 0x7fd4e81a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 -- 172.21.15.94:0/1716974412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4e8105b80 msgr2=0x7fd4e81a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4e8105b80 0x7fd4e81a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 -- 172.21.15.94:0/1716974412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 msgr2=0x7fd4e819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 0x7fd4e819a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 -- 172.21.15.94:0/1716974412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e81a1e10 con 0x7fd4e8103fd0 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4df400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 0x7fd4e819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.193+0000 7fd4e7e00640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8103fd0 0x7fd4e819a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800e9c0 tx=0x7fd4d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.194+0000 7fd4e5600640 1 -- 172.21.15.94:0/1716974412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800cd80 con 0x7fd4e8103fd0 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.194+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e81a2100 con 0x7fd4e8103fd0 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.194+0000 7fd4e5600640 1 -- 172.21.15.94:0/1716974412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4d80035f0 con 0x7fd4e8103fd0 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.194+0000 7fd4e5600640 1 -- 172.21.15.94:0/1716974412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8010690 con 0x7fd4e8103fd0 2024-04-25T14:59:16.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.194+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e81a25d0 con 0x7fd4e8103fd0 2024-04-25T14:59:16.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.196+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e810b340 con 0x7fd4e8103fd0 2024-04-25T14:59:16.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.197+0000 7fd4e5600640 1 -- 172.21.15.94:0/1716974412 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd4d8004000 con 0x7fd4e8103fd0 2024-04-25T14:59:16.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.199+0000 7fd4e5600640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4cc0a3bb0 0x7fd4cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:16.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.199+0000 7fd4e5600640 1 -- 172.21.15.94:0/1716974412 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd4d80d04f0 con 0x7fd4e8103fd0 2024-04-25T14:59:16.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.199+0000 7fd4df400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4cc0a3bb0 0x7fd4cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:16.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.200+0000 7fd4df400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4cc0a3bb0 0x7fd4cc0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0000c00 tx=0x7fd4d00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:16.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.206+0000 7fd4e5600640 1 -- 172.21.15.94:0/1716974412 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd4d809d2d0 con 0x7fd4e8103fd0 2024-04-25T14:59:16.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.454+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4e802d5c0 con 0x7fd4e8103fd0 2024-04-25T14:59:16.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.455+0000 7fd4e5600640 1 -- 172.21.15.94:0/1716974412 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fd4d809cbd0 con 0x7fd4e8103fd0 2024-04-25T14:59:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:16.457 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T14:59:16.457 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 4 2024-04-25T14:59:16.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4cc0a3bb0 msgr2=0x7fd4cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:16.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4cc0a3bb0 0x7fd4cc0a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0000c00 tx=0x7fd4d00314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8103fd0 msgr2=0x7fd4e819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8103fd0 0x7fd4e819a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800e9c0 tx=0x7fd4d800ee80 comp rx=0 tx=0).stop 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 shutdown_connections 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4cc0a3bb0 0x7fd4cc0a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4e8105b80 0x7fd4e81a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4e81051c0 0x7fd4e819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 --2- 172.21.15.94:0/1716974412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4e8103fd0 0x7fd4e819a290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 >> 172.21.15.94:0/1716974412 conn(0x7fd4e80ff560 msgr2=0x7fd4e8108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 shutdown_connections 2024-04-25T14:59:16.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:16.458+0000 7fd4ed400640 1 -- 172.21.15.94:0/1716974412 wait complete. 2024-04-25T14:59:16.710 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:16.711 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.711 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.711 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.711 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:16.711 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:16.711 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:16.711 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:16 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1716974412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:16 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1716974412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:16.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:16.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:16.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:16.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:16 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1716974412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:17 smithi137 ceph-mon[32439]: Deploying daemon mds.e on smithi137 2024-04-25T14:59:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:17 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.94:6836/1161222965,v1:172.21.15.94:6837/1161222965] up:boot 2024-04-25T14:59:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:17 smithi137 ceph-mon[32439]: fsmap 4 up:standby 2024-04-25T14:59:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:17 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:59:17.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:17 smithi138 ceph-mon[32875]: Deploying daemon mds.e on smithi137 2024-04-25T14:59:17.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:17 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.94:6836/1161222965,v1:172.21.15.94:6837/1161222965] up:boot 2024-04-25T14:59:17.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:17 smithi138 ceph-mon[32875]: fsmap 4 up:standby 2024-04-25T14:59:17.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:17 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:59:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:17 smithi094 ceph-mon[27862]: Deploying daemon mds.e on smithi137 2024-04-25T14:59:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:17 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.94:6836/1161222965,v1:172.21.15.94:6837/1161222965] up:boot 2024-04-25T14:59:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:17 smithi094 ceph-mon[27862]: fsmap 4 up:standby 2024-04-25T14:59:17.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:17 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T14:59:18.175 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:17 smithi137 systemd[1]: Starting Ceph mds.e for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:18.477 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 podman[62647]: 2024-04-25 14:59:18.077998644 +0000 UTC m=+0.017938473 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:18.477 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 podman[62647]: 2024-04-25 14:59:18.182840633 +0000 UTC m=+0.122780461 container create 513567bf6136bb7e26a12f86a986179c0204f8730e950403f7576c8bb1646bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T14:59:18.477 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 podman[62647]: 2024-04-25 14:59:18.376994638 +0000 UTC m=+0.316934466 container init 513567bf6136bb7e26a12f86a986179c0204f8730e950403f7576c8bb1646bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T14:59:18.477 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 podman[62647]: 2024-04-25 14:59:18.37984347 +0000 UTC m=+0.319783299 container start 513567bf6136bb7e26a12f86a986179c0204f8730e950403f7576c8bb1646bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:59:18.477 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 ceph-mds[62666]: -- 172.21.15.137:0/2715842181 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e98c3ba40 con 0x558e98c0ec00 2024-04-25T14:59:18.477 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e[62662]: starting mds.e at 2024-04-25T14:59:18.478 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 -- 172.21.15.94:0/3928292879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9810a160 msgr2=0x7fec981109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 --2- 172.21.15.94:0/3928292879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9810a160 0x7fec981109f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec90009ca0 tx=0x7fec9002a8c0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 -- 172.21.15.94:0/3928292879 shutdown_connections 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 --2- 172.21.15.94:0/3928292879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9810a160 0x7fec981109f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 --2- 172.21.15.94:0/3928292879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec981097a0 0x7fec98109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 --2- 172.21.15.94:0/3928292879 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 0x7fec981089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 -- 172.21.15.94:0/3928292879 >> 172.21.15.94:0/3928292879 conn(0x7fec98077570 msgr2=0x7fec98079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 -- 172.21.15.94:0/3928292879 shutdown_connections 2024-04-25T14:59:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.697+0000 7fec9e400640 1 -- 172.21.15.94:0/3928292879 wait complete. 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.698+0000 7fec9e400640 1 Processor -- start 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.698+0000 7fec9e400640 1 -- start start 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec9e400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 0x7fec9819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec9e400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec981097a0 0x7fec9819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec9e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec9810a160 0x7fec981a5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec9e400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fec98114110 con 0x7fec981097a0 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec9e400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fec98113f90 con 0x7fec981085b0 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec9e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec98114290 con 0x7fec9810a160 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 0x7fec9819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec8ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec981097a0 0x7fec9819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 0x7fec9819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:36124/0 (socket says 172.21.15.94:36124) 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec9810a160 0x7fec981a5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec8ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec981097a0 0x7fec9819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50698/0 (socket says 172.21.15.94:50698) 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97400640 1 -- 172.21.15.94:0/909597895 learned_addr learned my addr 172.21.15.94:0/909597895 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97400640 1 -- 172.21.15.94:0/909597895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec9810a160 msgr2=0x7fec981a5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec9810a160 0x7fec981a5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97400640 1 -- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec981097a0 msgr2=0x7fec9819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.699+0000 7fec97400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec981097a0 0x7fec9819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.700+0000 7fec97400640 1 -- 172.21.15.94:0/909597895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec981a6370 con 0x7fec981085b0 2024-04-25T14:59:18.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.700+0000 7fec97400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 0x7fec9819e750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e9c0 tx=0x7fec8800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:18.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.700+0000 7fec94c00640 1 -- 172.21.15.94:0/909597895 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8800cde0 con 0x7fec981085b0 2024-04-25T14:59:18.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.700+0000 7fec94c00640 1 -- 172.21.15.94:0/909597895 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec880108e0 con 0x7fec981085b0 2024-04-25T14:59:18.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.700+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec981a6660 con 0x7fec981085b0 2024-04-25T14:59:18.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.700+0000 7fec94c00640 1 -- 172.21.15.94:0/909597895 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec88018960 con 0x7fec981085b0 2024-04-25T14:59:18.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.700+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec981a6bb0 con 0x7fec981085b0 2024-04-25T14:59:18.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.701+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec9810f8d0 con 0x7fec981085b0 2024-04-25T14:59:18.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.703+0000 7fec94c00640 1 -- 172.21.15.94:0/909597895 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fec8801f020 con 0x7fec981085b0 2024-04-25T14:59:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.704+0000 7fec94c00640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.705+0000 7fec94c00640 1 -- 172.21.15.94:0/909597895 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fec880d0270 con 0x7fec981085b0 2024-04-25T14:59:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.705+0000 7fec8ea00640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.705+0000 7fec8ea00640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fec9819fc60 tx=0x7fec80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.709+0000 7fec94c00640 1 -- 172.21.15.94:0/909597895 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec88053ff0 con 0x7fec981085b0 2024-04-25T14:59:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:18 smithi137 ceph-mon[32439]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:18 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:18.732 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 bash[62647]: 513567bf6136bb7e26a12f86a986179c0204f8730e950403f7576c8bb1646bde 2024-04-25T14:59:18.733 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 14:59:18 smithi137 systemd[1]: Started Ceph mds.e for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:18.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.950+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec9802d5c0 con 0x7fec981085b0 2024-04-25T14:59:18.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.951+0000 7fec94c00640 1 -- 172.21.15.94:0/909597895 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7fec8809d050 con 0x7fec981085b0 2024-04-25T14:59:18.952 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:18.952 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T14:59:18.953 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 6 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fec7c0a3bb0 msgr2=0x7fec7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fec9819fc60 tx=0x7fec80005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 msgr2=0x7fec9819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 0x7fec9819e750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e9c0 tx=0x7fec8800ee80 comp rx=0 tx=0).stop 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 shutdown_connections 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fec7c0a3bb0 0x7fec7c0a6070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec9810a160 0x7fec981a5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec981097a0 0x7fec9819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 --2- 172.21.15.94:0/909597895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fec981085b0 0x7fec9819e750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.953+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 >> 172.21.15.94:0/909597895 conn(0x7fec98077570 msgr2=0x7fec9810caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.954+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 shutdown_connections 2024-04-25T14:59:18.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:18.954+0000 7fec9e400640 1 -- 172.21.15.94:0/909597895 wait complete. 2024-04-25T14:59:18.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:18 smithi138 ceph-mon[32875]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:18.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:18.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:18 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:18 smithi094 ceph-mon[27862]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:18 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.137:6834/1346240831,v1:172.21.15.137:6835/1346240831] up:boot 2024-04-25T14:59:19.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: fsmap 5 up:standby 2024-04-25T14:59:19.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:59:19.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:19.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:19.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:19.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:19.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: Deploying daemon mds.f on smithi138 2024-04-25T14:59:19.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:19 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/909597895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.137:6834/1346240831,v1:172.21.15.137:6835/1346240831] up:boot 2024-04-25T14:59:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: fsmap 5 up:standby 2024-04-25T14:59:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:59:19.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:19.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:19.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:19.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:19.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: Deploying daemon mds.f on smithi138 2024-04-25T14:59:19.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:19 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/909597895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.137:6834/1346240831,v1:172.21.15.137:6835/1346240831] up:boot 2024-04-25T14:59:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: fsmap 5 up:standby 2024-04-25T14:59:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T14:59:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:19.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: Deploying daemon mds.f on smithi138 2024-04-25T14:59:19.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:19 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/909597895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:20 smithi138 ceph-mon[32875]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:20.707 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:20 smithi138 systemd[1]: Starting Ceph mds.f for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:20.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:20 smithi137 ceph-mon[32439]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:20 smithi094 ceph-mon[27862]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:20.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:21.107 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:20 smithi138 podman[63385]: 2024-04-25 14:59:20.845794086 +0000 UTC m=+0.107005560 container create c20d22a287a8585f5c50f35213975203823068fd8ae37b7556ff4161155dcdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:59:21.107 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:20 smithi138 podman[63385]: 2024-04-25 14:59:20.756790586 +0000 UTC m=+0.018002031 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:21.107 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:21 smithi138 podman[63385]: 2024-04-25 14:59:21.08161551 +0000 UTC m=+0.342826938 container init c20d22a287a8585f5c50f35213975203823068fd8ae37b7556ff4161155dcdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:59:21.107 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:21 smithi138 podman[63385]: 2024-04-25 14:59:21.084056584 +0000 UTC m=+0.345268014 container start c20d22a287a8585f5c50f35213975203823068fd8ae37b7556ff4161155dcdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:59:21.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 -- 172.21.15.94:0/443231193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 msgr2=0x7ff8b4105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:21.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/443231193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 0x7ff8b4105de0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009860 tx=0x7ff8ac02a340 comp rx=0 tx=0).stop 2024-04-25T14:59:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 -- 172.21.15.94:0/443231193 shutdown_connections 2024-04-25T14:59:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/443231193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 0x7ff8b4105de0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/443231193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4102fb0 0x7ff8b4105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/443231193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8b4100640 0x7ff8b4102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 -- 172.21.15.94:0/443231193 >> 172.21.15.94:0/443231193 conn(0x7ff8b40fc220 msgr2=0x7ff8b40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 -- 172.21.15.94:0/443231193 shutdown_connections 2024-04-25T14:59:21.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.181+0000 7ff8b8c00640 1 -- 172.21.15.94:0/443231193 wait complete. 2024-04-25T14:59:21.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 Processor -- start 2024-04-25T14:59:21.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 -- start start 2024-04-25T14:59:21.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4100640 0x7ff8b419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:21.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8b4102fb0 0x7ff8b419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:21.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 0x7ff8b41a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:21.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4113f20 con 0x7ff8b4102fb0 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b4113da0 con 0x7ff8b4105960 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.182+0000 7ff8b8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b41140a0 con 0x7ff8b4100640 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4100640 0x7ff8b419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b2a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8b4102fb0 0x7ff8b419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4100640 0x7ff8b419e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50546/0 (socket says 172.21.15.94:50546) 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 -- 172.21.15.94:0/1750584537 learned_addr learned my addr 172.21.15.94:0/1750584537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3e00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 0x7ff8b41a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 -- 172.21.15.94:0/1750584537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 msgr2=0x7ff8b41a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 0x7ff8b41a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 -- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8b4102fb0 msgr2=0x7ff8b419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8b4102fb0 0x7ff8b419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 -- 172.21.15.94:0/1750584537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b41a6170 con 0x7ff8b4100640 2024-04-25T14:59:21.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8b3400640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4100640 0x7ff8b419e5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400efb0 tx=0x7ff8a400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:21.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.183+0000 7ff8abe00640 1 -- 172.21.15.94:0/1750584537 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a401b070 con 0x7ff8b4100640 2024-04-25T14:59:21.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.184+0000 7ff8abe00640 1 -- 172.21.15.94:0/1750584537 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8a400ba20 con 0x7ff8b4100640 2024-04-25T14:59:21.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.184+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b41a6460 con 0x7ff8b4100640 2024-04-25T14:59:21.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.184+0000 7ff8abe00640 1 -- 172.21.15.94:0/1750584537 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4016b30 con 0x7ff8b4100640 2024-04-25T14:59:21.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.184+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b41a6a00 con 0x7ff8b4100640 2024-04-25T14:59:21.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.185+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b40fdd10 con 0x7ff8b4100640 2024-04-25T14:59:21.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.187+0000 7ff8abe00640 1 -- 172.21.15.94:0/1750584537 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff8a401e070 con 0x7ff8b4100640 2024-04-25T14:59:21.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.188+0000 7ff8abe00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff8980a3c30 0x7ff8980a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:21.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.188+0000 7ff8abe00640 1 -- 172.21.15.94:0/1750584537 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff8a40d0200 con 0x7ff8b4100640 2024-04-25T14:59:21.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.188+0000 7ff8b2a00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff8980a3c30 0x7ff8980a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:21.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.189+0000 7ff8b2a00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff8980a3c30 0x7ff8980a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8b419fa60 tx=0x7ff8a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:21.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.190+0000 7ff8abe00640 1 -- 172.21.15.94:0/1750584537 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8a409c990 con 0x7ff8b4100640 2024-04-25T14:59:21.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.435+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8b402d5c0 con 0x7ff8b4100640 2024-04-25T14:59:21.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.436+0000 7ff8abe00640 1 -- 172.21.15.94:0/1750584537 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4702 (secure 0 0 0) 0x7ff8a40a0c90 con 0x7ff8b4100640 2024-04-25T14:59:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:21.438 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T14:59:21.438 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 6 2024-04-25T14:59:21.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff8980a3c30 msgr2=0x7ff8980a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff8980a3c30 0x7ff8980a60f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8b419fa60 tx=0x7ff8a0005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4100640 msgr2=0x7ff8b419e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4100640 0x7ff8b419e5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400efb0 tx=0x7ff8a400c800 comp rx=0 tx=0).stop 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 shutdown_connections 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff8980a3c30 0x7ff8980a60f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff8b4105960 0x7ff8b41a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff8b4102fb0 0x7ff8b419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 --2- 172.21.15.94:0/1750584537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8b4100640 0x7ff8b419e5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 >> 172.21.15.94:0/1750584537 conn(0x7ff8b40fc220 msgr2=0x7ff8b41038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 shutdown_connections 2024-04-25T14:59:21.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:21.439+0000 7ff8b8c00640 1 -- 172.21.15.94:0/1750584537 wait complete. 2024-04-25T14:59:21.456 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:21 smithi138 ceph-mds[63404]: -- 172.21.15.138:0/2457808003 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56139a5dda40 con 0x56139a5b0800 2024-04-25T14:59:21.457 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f[63400]: starting mds.f at 2024-04-25T14:59:21.457 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:21 smithi138 bash[63385]: c20d22a287a8585f5c50f35213975203823068fd8ae37b7556ff4161155dcdcb 2024-04-25T14:59:21.457 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 14:59:21 smithi138 systemd[1]: Started Ceph mds.f for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: Deploying daemon mds.c on smithi138 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1750584537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:22.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.138:6834/4151262699,v1:172.21.15.138:6835/4151262699] up:boot 2024-04-25T14:59:22.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: fsmap 6 up:standby 2024-04-25T14:59:22.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:22 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:59:22.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: Deploying daemon mds.c on smithi138 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1750584537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.138:6834/4151262699,v1:172.21.15.138:6835/4151262699] up:boot 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: fsmap 6 up:standby 2024-04-25T14:59:22.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:22 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:59:22.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:22.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:22.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:22.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:22.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: Deploying daemon mds.c on smithi138 2024-04-25T14:59:22.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1750584537' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:22.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:22.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.138:6834/4151262699,v1:172.21.15.138:6835/4151262699] up:boot 2024-04-25T14:59:22.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: fsmap 6 up:standby 2024-04-25T14:59:22.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:22 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T14:59:23.457 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 systemd[1]: Starting Ceph mds.c for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:23.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:23.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 -- 172.21.15.94:0/4276483001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc103970 msgr2=0x7f88cc10a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:23.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 --2- 172.21.15.94:0/4276483001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc103970 0x7f88cc10a200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009ca0 tx=0x7f88c802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 -- 172.21.15.94:0/4276483001 shutdown_connections 2024-04-25T14:59:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 --2- 172.21.15.94:0/4276483001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc103970 0x7f88cc10a200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 --2- 172.21.15.94:0/4276483001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88cc102fb0 0x7f88cc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 --2- 172.21.15.94:0/4276483001 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc101dc0 0x7f88cc1021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 -- 172.21.15.94:0/4276483001 >> 172.21.15.94:0/4276483001 conn(0x7f88cc0fd330 msgr2=0x7f88cc0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 -- 172.21.15.94:0/4276483001 shutdown_connections 2024-04-25T14:59:23.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.671+0000 7f88d3200640 1 -- 172.21.15.94:0/4276483001 wait complete. 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 Processor -- start 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 -- start start 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc101dc0 0x7f88cc19a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc102fb0 0x7f88cc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88cc103970 0x7f88cc1a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc10d8f0 con 0x7f88cc102fb0 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc10d770 con 0x7f88cc101dc0 2024-04-25T14:59:23.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.672+0000 7f88d3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc10da70 con 0x7f88cc103970 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc102fb0 0x7f88cc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc101dc0 0x7f88cc19a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc102fb0 0x7f88cc19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50752/0 (socket says 172.21.15.94:50752) 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1400640 1 -- 172.21.15.94:0/3617589093 learned_addr learned my addr 172.21.15.94:0/3617589093 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d2800640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88cc103970 0x7f88cc1a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1e00640 1 -- 172.21.15.94:0/3617589093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88cc103970 msgr2=0x7f88cc1a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1e00640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88cc103970 0x7f88cc1a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1e00640 1 -- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc102fb0 msgr2=0x7f88cc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1e00640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc102fb0 0x7f88cc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1e00640 1 -- 172.21.15.94:0/3617589093 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc1a1dd0 con 0x7f88cc101dc0 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1400640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc102fb0 0x7f88cc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.673+0000 7f88d1e00640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc101dc0 0x7f88cc19a1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e9c0 tx=0x7f88c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:23.676 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.674+0000 7f88c6a00640 1 -- 172.21.15.94:0/3617589093 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c000cd80 con 0x7f88cc101dc0 2024-04-25T14:59:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.674+0000 7f88c6a00640 1 -- 172.21.15.94:0/3617589093 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c00108e0 con 0x7f88cc101dc0 2024-04-25T14:59:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.674+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc1a20c0 con 0x7f88cc101dc0 2024-04-25T14:59:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.674+0000 7f88c6a00640 1 -- 172.21.15.94:0/3617589093 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0018900 con 0x7f88cc101dc0 2024-04-25T14:59:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.674+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc1a2610 con 0x7f88cc101dc0 2024-04-25T14:59:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.675+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88cc1a2870 con 0x7f88cc101dc0 2024-04-25T14:59:23.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.677+0000 7f88c6a00640 1 -- 172.21.15.94:0/3617589093 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f88c001f020 con 0x7f88cc101dc0 2024-04-25T14:59:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.678+0000 7f88c6a00640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88b40a3bb0 0x7f88b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.678+0000 7f88c6a00640 1 -- 172.21.15.94:0/3617589093 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f88c00cff80 con 0x7f88cc101dc0 2024-04-25T14:59:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.679+0000 7f88d1400640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88b40a3bb0 0x7f88b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:23.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.679+0000 7f88d1400640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88b40a3bb0 0x7f88b40a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88cc19b6c0 tx=0x7f88bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:23.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.683+0000 7f88c6a00640 1 -- 172.21.15.94:0/3617589093 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88c009cd60 con 0x7f88cc101dc0 2024-04-25T14:59:23.886 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 podman[63693]: 2024-04-25 14:59:23.475498919 +0000 UTC m=+0.018086363 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:23.886 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 podman[63693]: 2024-04-25 14:59:23.591446678 +0000 UTC m=+0.134034132 container create b76cdd18b0ed3e6e7912ecd129945d0301eab02e87c693fd37a734eb5c094545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T14:59:23.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.924+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88cc1a2a40 con 0x7f88cc101dc0 2024-04-25T14:59:23.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.925+0000 7f88c6a00640 1 -- 172.21.15.94:0/3617589093 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f88c009c660 con 0x7f88cc101dc0 2024-04-25T14:59:23.927 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:23.927 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T14:59:23.927 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88b40a3bb0 msgr2=0x7f88b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88b40a3bb0 0x7f88b40a6070 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f88cc19b6c0 tx=0x7f88bc005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc101dc0 msgr2=0x7f88cc19a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc101dc0 0x7f88cc19a1b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88c000e9c0 tx=0x7f88c000ee80 comp rx=0 tx=0).stop 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 shutdown_connections 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88b40a3bb0 0x7f88b40a6070 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88cc103970 0x7f88cc1a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88cc102fb0 0x7f88cc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 --2- 172.21.15.94:0/3617589093 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88cc101dc0 0x7f88cc19a1b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:23.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.928+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 >> 172.21.15.94:0/3617589093 conn(0x7f88cc0fd330 msgr2=0x7f88cc074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:23.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.929+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 shutdown_connections 2024-04-25T14:59:23.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:23.929+0000 7f88d3200640 1 -- 172.21.15.94:0/3617589093 wait complete. 2024-04-25T14:59:24.207 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 podman[63693]: 2024-04-25 14:59:23.885501257 +0000 UTC m=+0.428088679 container init b76cdd18b0ed3e6e7912ecd129945d0301eab02e87c693fd37a734eb5c094545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-25T14:59:24.207 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 podman[63693]: 2024-04-25 14:59:23.888268598 +0000 UTC m=+0.430856011 container start b76cdd18b0ed3e6e7912ecd129945d0301eab02e87c693fd37a734eb5c094545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T14:59:24.207 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 ceph-mds[63712]: -- 172.21.15.138:0/3606695627 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bbb63ba40 con 0x563bbb60e800 2024-04-25T14:59:24.207 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c[63708]: starting mds.c at 2024-04-25T14:59:24.207 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 bash[63693]: b76cdd18b0ed3e6e7912ecd129945d0301eab02e87c693fd37a734eb5c094545 2024-04-25T14:59:24.207 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 14:59:23 smithi138 systemd[1]: Started Ceph mds.c for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3617589093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:24.877 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:24 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3617589093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:24 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3617589093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:24.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:24.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:24.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:24 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:25.850 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:25 smithi137 ceph-mon[32439]: Deploying daemon mds.h on smithi137 2024-04-25T14:59:25.850 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:25 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.138:6836/4267251536,v1:172.21.15.138:6837/4267251536] up:boot 2024-04-25T14:59:25.850 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:25 smithi137 ceph-mon[32439]: fsmap 7 up:standby 2024-04-25T14:59:25.850 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:25 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:59:25.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:25.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:25 smithi138 ceph-mon[32875]: Deploying daemon mds.h on smithi137 2024-04-25T14:59:25.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:25 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.138:6836/4267251536,v1:172.21.15.138:6837/4267251536] up:boot 2024-04-25T14:59:25.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:25 smithi138 ceph-mon[32875]: fsmap 7 up:standby 2024-04-25T14:59:25.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:25 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:59:25.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:25 smithi094 ceph-mon[27862]: Deploying daemon mds.h on smithi137 2024-04-25T14:59:25.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:25 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.138:6836/4267251536,v1:172.21.15.138:6837/4267251536] up:boot 2024-04-25T14:59:25.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:25 smithi094 ceph-mon[27862]: fsmap 7 up:standby 2024-04-25T14:59:25.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:25 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T14:59:26.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 -- 172.21.15.94:0/3750738187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4107ab0 msgr2=0x7fe5c4109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/3750738187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4107ab0 0x7fe5c4109f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009ca0 tx=0x7fe5bc02a8c0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 -- 172.21.15.94:0/3750738187 shutdown_connections 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/3750738187 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4107ab0 0x7fe5c4109f00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/3750738187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c4105180 0x7fe5c4107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/3750738187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4102850 0x7fe5c4104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 -- 172.21.15.94:0/3750738187 >> 172.21.15.94:0/3750738187 conn(0x7fe5c40fc220 msgr2=0x7fe5c40fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 -- 172.21.15.94:0/3750738187 shutdown_connections 2024-04-25T14:59:26.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.161+0000 7fe5c9a00640 1 -- 172.21.15.94:0/3750738187 wait complete. 2024-04-25T14:59:26.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 Processor -- start 2024-04-25T14:59:26.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 -- start start 2024-04-25T14:59:26.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c4102850 0x7fe5c419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:26.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4105180 0x7fe5c419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:26.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4107ab0 0x7fe5c41a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410d950 con 0x7fe5c4102850 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410d7d0 con 0x7fe5c4105180 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.162+0000 7fe5c9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c410dad0 con 0x7fe5c4107ab0 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.163+0000 7fe5c9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4107ab0 0x7fe5c41a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.163+0000 7fe5c9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4107ab0 0x7fe5c41a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:50586/0 (socket says 172.21.15.94:50586) 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.163+0000 7fe5c9000640 1 -- 172.21.15.94:0/1457873560 learned_addr learned my addr 172.21.15.94:0/1457873560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.163+0000 7fe5c3e00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c4102850 0x7fe5c419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:26.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.163+0000 7fe5c3400640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4105180 0x7fe5c419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.163+0000 7fe5c9000640 1 -- 172.21.15.94:0/1457873560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4105180 msgr2=0x7fe5c419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.163+0000 7fe5c9000640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4105180 0x7fe5c419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c9000640 1 -- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c4102850 msgr2=0x7fe5c419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c9000640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c4102850 0x7fe5c419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c9000640 1 -- 172.21.15.94:0/1457873560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c41a1e20 con 0x7fe5c4107ab0 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c9000640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4107ab0 0x7fe5c41a17b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc033040 tx=0x7fe5bc0316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c0c00640 1 -- 172.21.15.94:0/1457873560 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc003b70 con 0x7fe5c4107ab0 2024-04-25T14:59:26.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c41a20b0 con 0x7fe5c4107ab0 2024-04-25T14:59:26.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c0c00640 1 -- 172.21.15.94:0/1457873560 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5bc003d10 con 0x7fe5c4107ab0 2024-04-25T14:59:26.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c0c00640 1 -- 172.21.15.94:0/1457873560 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc02fb40 con 0x7fe5c4107ab0 2024-04-25T14:59:26.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.164+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c41a2650 con 0x7fe5c4107ab0 2024-04-25T14:59:26.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.165+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c41095a0 con 0x7fe5c4107ab0 2024-04-25T14:59:26.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.168+0000 7fe5c0c00640 1 -- 172.21.15.94:0/1457873560 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5bc004610 con 0x7fe5c4107ab0 2024-04-25T14:59:26.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.169+0000 7fe5c0c00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe5a80a3bb0 0x7fe5a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:26.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.169+0000 7fe5c0c00640 1 -- 172.21.15.94:0/1457873560 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe5bc0f1570 con 0x7fe5c4107ab0 2024-04-25T14:59:26.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.169+0000 7fe5c3e00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe5a80a3bb0 0x7fe5a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.170+0000 7fe5c3e00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe5a80a3bb0 0x7fe5a80a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009bd0 tx=0x7fe5b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.171+0000 7fe5c0c00640 1 -- 172.21.15.94:0/1457873560 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5bc0be350 con 0x7fe5c4107ab0 2024-04-25T14:59:26.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.412+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5c402d7d0 con 0x7fe5c4107ab0 2024-04-25T14:59:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.414+0000 7fe5c0c00640 1 -- 172.21.15.94:0/1457873560 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6358 (secure 0 0 0) 0x7fe5bc0bdc50 con 0x7fe5c4107ab0 2024-04-25T14:59:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:26.415 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T14:59:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe5a80a3bb0 msgr2=0x7fe5a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe5a80a3bb0 0x7fe5a80a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4009bd0 tx=0x7fe5b4005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4107ab0 msgr2=0x7fe5c41a17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4107ab0 0x7fe5c41a17b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc033040 tx=0x7fe5bc0316f0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 shutdown_connections 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe5a80a3bb0 0x7fe5a80a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5c4107ab0 0x7fe5c41a17b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe5c4105180 0x7fe5c419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 --2- 172.21.15.94:0/1457873560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe5c4102850 0x7fe5c419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.416+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 >> 172.21.15.94:0/1457873560 conn(0x7fe5c40fc220 msgr2=0x7fe5c4101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.417+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 shutdown_connections 2024-04-25T14:59:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:26.417+0000 7fe5c9a00640 1 -- 172.21.15.94:0/1457873560 wait complete. 2024-04-25T14:59:26.482 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:59:26 smithi137 podman[62954]: 2024-04-25 14:59:26.444120481 +0000 UTC m=+0.129677663 container create 4976fcf0fff1e83c0abf055796c208fbefeacbe2ece0410ba159774b5182e51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T14:59:26.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:26 smithi094 ceph-mon[27862]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:26.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:26 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1457873560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:26 smithi138 ceph-mon[32875]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:26 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1457873560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:26 smithi137 ceph-mon[32439]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:26 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1457873560' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:26.982 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:59:26 smithi137 podman[62954]: 2024-04-25 14:59:26.697468963 +0000 UTC m=+0.383026146 container init 4976fcf0fff1e83c0abf055796c208fbefeacbe2ece0410ba159774b5182e51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T14:59:26.983 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:59:26 smithi137 podman[62954]: 2024-04-25 14:59:26.70000185 +0000 UTC m=+0.385559033 container start 4976fcf0fff1e83c0abf055796c208fbefeacbe2ece0410ba159774b5182e51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T14:59:26.983 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:59:26 smithi137 ceph-mds[62973]: -- 172.21.15.137:0/4195586128 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d5dc23a40 con 0x562d5dbf7000 2024-04-25T14:59:26.983 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:59:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h[62969]: starting mds.h at 2024-04-25T14:59:26.983 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:59:26 smithi137 bash[62954]: 4976fcf0fff1e83c0abf055796c208fbefeacbe2ece0410ba159774b5182e51f 2024-04-25T14:59:26.983 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 14:59:26 smithi137 systemd[1]: Started Ceph mds.h for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: Deploying daemon mds.i on smithi138 2024-04-25T14:59:28.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:28.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.137:6836/4197829661,v1:172.21.15.137:6837/4197829661] up:boot 2024-04-25T14:59:28.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: fsmap 8 up:standby 2024-04-25T14:59:28.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:59:28.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:27 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:59:28.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: Deploying daemon mds.i on smithi138 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.137:6836/4197829661,v1:172.21.15.137:6837/4197829661] up:boot 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: fsmap 8 up:standby 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:59:28.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:27 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:59:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:28.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: Deploying daemon mds.i on smithi138 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.137:6836/4197829661,v1:172.21.15.137:6837/4197829661] up:boot 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: fsmap 8 up:standby 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:59:28.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:27 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T14:59:28.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:28.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 -- 172.21.15.94:0/2518544254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 msgr2=0x7f68a8101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2518544254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a8101e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009900 tx=0x7f68a002a410 comp rx=0 tx=0).stop 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 -- 172.21.15.94:0/2518544254 shutdown_connections 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2518544254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a8101e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2518544254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a8101010 0x7f68a8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2518544254 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68a81006b0 0x7f68a8100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 -- 172.21.15.94:0/2518544254 >> 172.21.15.94:0/2518544254 conn(0x7f68a80fc2f0 msgr2=0x7f68a80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 -- 172.21.15.94:0/2518544254 shutdown_connections 2024-04-25T14:59:28.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.651+0000 7f68a7e00640 1 -- 172.21.15.94:0/2518544254 wait complete. 2024-04-25T14:59:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.652+0000 7f68a7e00640 1 Processor -- start 2024-04-25T14:59:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.652+0000 7f68a7e00640 1 -- start start 2024-04-25T14:59:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68a81006b0 0x7f68a819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a8101010 0x7f68a819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:28.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a81a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f68a810fb20 con 0x7f68a81019b0 2024-04-25T14:59:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f68a810f9a0 con 0x7f68a81006b0 2024-04-25T14:59:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68a810fca0 con 0x7f68a8101010 2024-04-25T14:59:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a81a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:28.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a81a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49860/0 (socket says 172.21.15.94:49860) 2024-04-25T14:59:28.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7400640 1 -- 172.21.15.94:0/2516936243 learned_addr learned my addr 172.21.15.94:0/2516936243 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:28.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f689e000640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a8101010 0x7f68a819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:28.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a6a00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68a81006b0 0x7f68a819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:28.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f689e000640 1 -- 172.21.15.94:0/2516936243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68a81006b0 msgr2=0x7f68a819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:28.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f689e000640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68a81006b0 0x7f68a819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f689e000640 1 -- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 msgr2=0x7f68a81a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:28.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f689e000640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a81a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f689e000640 1 -- 172.21.15.94:0/2516936243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a81a1dd0 con 0x7f68a8101010 2024-04-25T14:59:28.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.653+0000 7f68a7400640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:28.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.654+0000 7f689e000640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a8101010 0x7f68a819a6f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f689000e9f0 tx=0x7f689000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:28.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.654+0000 7f68aca00640 1 -- 172.21.15.94:0/2516936243 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689000ccc0 con 0x7f68a8101010 2024-04-25T14:59:28.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.654+0000 7f68aca00640 1 -- 172.21.15.94:0/2516936243 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6890005020 con 0x7f68a8101010 2024-04-25T14:59:28.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.654+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a81a20c0 con 0x7f68a8101010 2024-04-25T14:59:28.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.654+0000 7f68aca00640 1 -- 172.21.15.94:0/2516936243 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6890004a00 con 0x7f68a8101010 2024-04-25T14:59:28.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.654+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a81a25c0 con 0x7f68a8101010 2024-04-25T14:59:28.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.655+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a810b400 con 0x7f68a8101010 2024-04-25T14:59:28.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.657+0000 7f68aca00640 1 -- 172.21.15.94:0/2516936243 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6890004ba0 con 0x7f68a8101010 2024-04-25T14:59:28.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.658+0000 7f68aca00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f688c0a3bb0 0x7f688c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:28.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.659+0000 7f68aca00640 1 -- 172.21.15.94:0/2516936243 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f68900d00f0 con 0x7f68a8101010 2024-04-25T14:59:28.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.659+0000 7f68a6a00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f688c0a3bb0 0x7f688c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:28.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.660+0000 7f68a6a00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f688c0a3bb0 0x7f688c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6898005fd0 tx=0x7f6898009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:28.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.661+0000 7f68aca00640 1 -- 172.21.15.94:0/2516936243 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f689009ced0 con 0x7f68a8101010 2024-04-25T14:59:28.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.903+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68a802d810 con 0x7f68a8101010 2024-04-25T14:59:28.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.904+0000 7f68aca00640 1 -- 172.21.15.94:0/2516936243 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f689009c7d0 con 0x7f68a8101010 2024-04-25T14:59:28.906 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:28.906 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T14:59:28.906 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f688c0a3bb0 msgr2=0x7f688c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f688c0a3bb0 0x7f688c0a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6898005fd0 tx=0x7f6898009500 comp rx=0 tx=0).stop 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a8101010 msgr2=0x7f68a819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a8101010 0x7f68a819a6f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f689000e9f0 tx=0x7f689000eeb0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 shutdown_connections 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f688c0a3bb0 0x7f688c0a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68a81019b0 0x7f68a81a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68a8101010 0x7f68a819a6f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 --2- 172.21.15.94:0/2516936243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f68a81006b0 0x7f68a819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:28.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.907+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 >> 172.21.15.94:0/2516936243 conn(0x7f68a80fc2f0 msgr2=0x7f68a80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:28.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.908+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 shutdown_connections 2024-04-25T14:59:28.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:28.908+0000 7f68a7e00640 1 -- 172.21.15.94:0/2516936243 wait complete. 2024-04-25T14:59:29.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:28 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2516936243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:29.207 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 systemd[1]: Starting Ceph mds.i for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:29.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:28 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2516936243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:29.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:28 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2516936243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:29.566 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 podman[64003]: 2024-04-25 14:59:29.206707664 +0000 UTC m=+0.017963203 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:29.566 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 podman[64003]: 2024-04-25 14:59:29.320916231 +0000 UTC m=+0.132171803 container create 84824fb92537d9fe8997dbba64a46327d5f3a4c657974814ad3a5740b979c12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T14:59:29.567 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 podman[64003]: 2024-04-25 14:59:29.540093893 +0000 UTC m=+0.351349423 container init 84824fb92537d9fe8997dbba64a46327d5f3a4c657974814ad3a5740b979c12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T14:59:29.567 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 podman[64003]: 2024-04-25 14:59:29.542808985 +0000 UTC m=+0.354064516 container start 84824fb92537d9fe8997dbba64a46327d5f3a4c657974814ad3a5740b979c12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T14:59:29.957 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mds[64023]: -- 172.21.15.138:0/74944767 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2c16bfa40 con 0x55c2c1692c00 2024-04-25T14:59:29.957 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i[64019]: starting mds.i at 2024-04-25T14:59:29.957 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 bash[64003]: 84824fb92537d9fe8997dbba64a46327d5f3a4c657974814ad3a5740b979c12b 2024-04-25T14:59:29.957 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 14:59:29 smithi138 systemd[1]: Started Ceph mds.i for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:29 smithi137 ceph-mon[32439]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:30.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:30.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:29 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:30.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mon[32875]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:30.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:30.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:30.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:29 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:30.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:29 smithi094 ceph-mon[27862]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:30.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:30.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:30.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:30.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:29 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:30.928 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:31.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:30 smithi094 ceph-mon[27862]: Deploying daemon mds.g on smithi094 2024-04-25T14:59:31.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:30 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.138:6838/593747621,v1:172.21.15.138:6839/593747621] up:boot 2024-04-25T14:59:31.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:30 smithi094 ceph-mon[27862]: fsmap 9 up:standby 2024-04-25T14:59:31.110 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:30 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:59:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.129+0000 7fa058600640 1 -- 172.21.15.94:0/1348258205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0501041a0 msgr2=0x7fa0501045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.129+0000 7fa058600640 1 --2- 172.21.15.94:0/1348258205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0501041a0 0x7fa0501045c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa044009900 tx=0x7fa04402a410 comp rx=0 tx=0).stop 2024-04-25T14:59:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.130+0000 7fa058600640 1 -- 172.21.15.94:0/1348258205 shutdown_connections 2024-04-25T14:59:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.130+0000 7fa058600640 1 --2- 172.21.15.94:0/1348258205 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa050105d50 0x7fa05010c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.130+0000 7fa058600640 1 --2- 172.21.15.94:0/1348258205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa050105390 0x7fa050105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.130+0000 7fa058600640 1 --2- 172.21.15.94:0/1348258205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0501041a0 0x7fa0501045c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.130+0000 7fa058600640 1 -- 172.21.15.94:0/1348258205 >> 172.21.15.94:0/1348258205 conn(0x7fa0500ff710 msgr2=0x7fa050101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.130+0000 7fa058600640 1 -- 172.21.15.94:0/1348258205 shutdown_connections 2024-04-25T14:59:31.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.130+0000 7fa058600640 1 -- 172.21.15.94:0/1348258205 wait complete. 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 Processor -- start 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 -- start start 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0501041a0 0x7fa05019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa050105390 0x7fa05019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa050105d50 0x7fa0501a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010fce0 con 0x7fa050105d50 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010fb60 con 0x7fa050105390 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa058600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa05010fe60 con 0x7fa0501041a0 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa056600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa050105d50 0x7fa0501a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa055200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa050105390 0x7fa05019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa056600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa050105d50 0x7fa0501a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49878/0 (socket says 172.21.15.94:49878) 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa056600640 1 -- 172.21.15.94:0/1268463323 learned_addr learned my addr 172.21.15.94:0/1268463323 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:31.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.131+0000 7fa055c00640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0501041a0 0x7fa05019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa055200640 1 -- 172.21.15.94:0/1268463323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0501041a0 msgr2=0x7fa05019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa055200640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0501041a0 0x7fa05019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa055200640 1 -- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa050105d50 msgr2=0x7fa0501a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa055200640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa050105d50 0x7fa0501a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa055200640 1 -- 172.21.15.94:0/1268463323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0501a2020 con 0x7fa050105390 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa056600640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa050105d50 0x7fa0501a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa055200640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa050105390 0x7fa05019a940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9f0 tx=0x7fa03c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa04e000640 1 -- 172.21.15.94:0/1268463323 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00ccc0 con 0x7fa050105390 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.132+0000 7fa04e000640 1 -- 172.21.15.94:0/1268463323 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa03c005020 con 0x7fa050105390 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.133+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0501a2310 con 0x7fa050105390 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.133+0000 7fa04e000640 1 -- 172.21.15.94:0/1268463323 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c004a00 con 0x7fa050105390 2024-04-25T14:59:31.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.133+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0501a2860 con 0x7fa050105390 2024-04-25T14:59:31.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.134+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05010b630 con 0x7fa050105390 2024-04-25T14:59:31.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.136+0000 7fa04e000640 1 -- 172.21.15.94:0/1268463323 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa03c004ba0 con 0x7fa050105390 2024-04-25T14:59:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.138+0000 7fa04e000640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa0380a3c30 0x7fa0380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.138+0000 7fa04e000640 1 -- 172.21.15.94:0/1268463323 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa03c0cfee0 con 0x7fa050105390 2024-04-25T14:59:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.138+0000 7fa055c00640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa0380a3c30 0x7fa0380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:31.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.139+0000 7fa055c00640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa0380a3c30 0x7fa0380a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa044009230 tx=0x7fa0440314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:31.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.142+0000 7fa04e000640 1 -- 172.21.15.94:0/1268463323 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa03c09ccc0 con 0x7fa050105390 2024-04-25T14:59:31.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:30 smithi137 ceph-mon[32439]: Deploying daemon mds.g on smithi094 2024-04-25T14:59:31.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:30 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.138:6838/593747621,v1:172.21.15.138:6839/593747621] up:boot 2024-04-25T14:59:31.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:30 smithi137 ceph-mon[32439]: fsmap 9 up:standby 2024-04-25T14:59:31.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:30 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:59:31.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.402+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa050062b50 con 0x7fa050105390 2024-04-25T14:59:31.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.404+0000 7fa04e000640 1 -- 172.21.15.94:0/1268463323 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8013 (secure 0 0 0) 0x7fa03c09c5c0 con 0x7fa050105390 2024-04-25T14:59:31.405 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:31.405 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T14:59:31.406 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 10 2024-04-25T14:59:31.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa0380a3c30 msgr2=0x7fa0380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa0380a3c30 0x7fa0380a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa044009230 tx=0x7fa0440314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa050105390 msgr2=0x7fa05019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa050105390 0x7fa05019a940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9f0 tx=0x7fa03c00eeb0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 shutdown_connections 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa0380a3c30 0x7fa0380a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa050105d50 0x7fa0501a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa050105390 0x7fa05019a940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 --2- 172.21.15.94:0/1268463323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0501041a0 0x7fa05019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.406+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 >> 172.21.15.94:0/1268463323 conn(0x7fa0500ff710 msgr2=0x7fa050108800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.407+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 shutdown_connections 2024-04-25T14:59:31.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:31.407+0000 7fa058600640 1 -- 172.21.15.94:0/1268463323 wait complete. 2024-04-25T14:59:31.456 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:30 smithi138 ceph-mon[32875]: Deploying daemon mds.g on smithi094 2024-04-25T14:59:31.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:30 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.138:6838/593747621,v1:172.21.15.138:6839/593747621] up:boot 2024-04-25T14:59:31.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:30 smithi138 ceph-mon[32875]: fsmap 9 up:standby 2024-04-25T14:59:31.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:30 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T14:59:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:31 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1268463323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:31 smithi137 ceph-mon[32439]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:32.427 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:31 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1268463323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:32.428 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:31 smithi094 ceph-mon[27862]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:32.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:31 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1268463323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:32.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:31 smithi138 ceph-mon[32875]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:33.099 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:32 smithi094 systemd[1]: Starting Ceph mds.g for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:33.385 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 podman[69120]: 2024-04-25 14:59:33.096362072 +0000 UTC m=+0.124341557 container create 48077639c94e2e02c1a572b4f7a33fc1b80c418149f094115e99b5fd424638d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:59:33.385 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 podman[69120]: 2024-04-25 14:59:33.00758341 +0000 UTC m=+0.035562860 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:33.386 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 podman[69120]: 2024-04-25 14:59:33.384028494 +0000 UTC m=+0.412008022 container init 48077639c94e2e02c1a572b4f7a33fc1b80c418149f094115e99b5fd424638d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:59:33.428 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.626+0000 7ff5e2600640 1 -- 172.21.15.94:0/942502134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc1041d0 msgr2=0x7ff5dc1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.626+0000 7ff5e2600640 1 --2- 172.21.15.94:0/942502134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc1041d0 0x7ff5dc1045f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009900 tx=0x7ff5cc02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.626+0000 7ff5e2600640 1 -- 172.21.15.94:0/942502134 shutdown_connections 2024-04-25T14:59:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.626+0000 7ff5e2600640 1 --2- 172.21.15.94:0/942502134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc105d80 0x7ff5dc10c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.626+0000 7ff5e2600640 1 --2- 172.21.15.94:0/942502134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5dc1053c0 0x7ff5dc105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.626+0000 7ff5e2600640 1 --2- 172.21.15.94:0/942502134 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc1041d0 0x7ff5dc1045f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.626+0000 7ff5e2600640 1 -- 172.21.15.94:0/942502134 >> 172.21.15.94:0/942502134 conn(0x7ff5dc0ff740 msgr2=0x7ff5dc101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.627+0000 7ff5e2600640 1 -- 172.21.15.94:0/942502134 shutdown_connections 2024-04-25T14:59:33.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.627+0000 7ff5e2600640 1 -- 172.21.15.94:0/942502134 wait complete. 2024-04-25T14:59:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 Processor -- start 2024-04-25T14:59:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 -- start start 2024-04-25T14:59:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5dc1041d0 0x7ff5dc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1053c0 0x7ff5dc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:33.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc105d80 0x7ff5dc1a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc10fd60 con 0x7ff5dc1053c0 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc10fbe0 con 0x7ff5dc105d80 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.628+0000 7ff5e2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5dc10fee0 con 0x7ff5dc1041d0 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc105d80 0x7ff5dc1a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc105d80 0x7ff5dc1a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:45608/0 (socket says 172.21.15.94:45608) 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 -- 172.21.15.94:0/1121883991 learned_addr learned my addr 172.21.15.94:0/1121883991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5db400640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5dc1041d0 0x7ff5dc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5daa00640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1053c0 0x7ff5dc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 -- 172.21.15.94:0/1121883991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5dc1041d0 msgr2=0x7ff5dc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5dc1041d0 0x7ff5dc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 -- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1053c0 msgr2=0x7ff5dc19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1053c0 0x7ff5dc19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 -- 172.21.15.94:0/1121883991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5dc1a2060 con 0x7ff5dc105d80 2024-04-25T14:59:33.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5dbe00640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc105d80 0x7ff5dc1a19f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400eb10 tx=0x7ff5d400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.629+0000 7ff5d3e00640 1 -- 172.21.15.94:0/1121883991 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4004b20 con 0x7ff5dc105d80 2024-04-25T14:59:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.630+0000 7ff5d3e00640 1 -- 172.21.15.94:0/1121883991 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5d4004cc0 con 0x7ff5dc105d80 2024-04-25T14:59:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.630+0000 7ff5d3e00640 1 -- 172.21.15.94:0/1121883991 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4010670 con 0x7ff5dc105d80 2024-04-25T14:59:33.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.630+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5dc1a2350 con 0x7ff5dc105d80 2024-04-25T14:59:33.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.630+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5dc1a28a0 con 0x7ff5dc105d80 2024-04-25T14:59:33.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.631+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5dc10b560 con 0x7ff5dc105d80 2024-04-25T14:59:33.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.633+0000 7ff5d3e00640 1 -- 172.21.15.94:0/1121883991 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff5d4005020 con 0x7ff5dc105d80 2024-04-25T14:59:33.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.634+0000 7ff5d3e00640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff5c00a3c30 0x7ff5c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:33.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.635+0000 7ff5d3e00640 1 -- 172.21.15.94:0/1121883991 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff5d40d0530 con 0x7ff5dc105d80 2024-04-25T14:59:33.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.635+0000 7ff5db400640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff5c00a3c30 0x7ff5c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:33.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.636+0000 7ff5db400640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff5c00a3c30 0x7ff5c00a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005ab0 tx=0x7ff5cc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:33.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.638+0000 7ff5d3e00640 1 -- 172.21.15.94:0/1121883991 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5d409d310 con 0x7ff5dc105d80 2024-04-25T14:59:33.673 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 podman[69120]: 2024-04-25 14:59:33.393969957 +0000 UTC m=+0.421949451 container start 48077639c94e2e02c1a572b4f7a33fc1b80c418149f094115e99b5fd424638d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T14:59:33.673 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 ceph-mds[69140]: -- 172.21.15.94:0/3032133005 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649a1277a40 con 0x5649a124a800 2024-04-25T14:59:33.673 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g[69135]: starting mds.g at 2024-04-25T14:59:33.673 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 bash[69120]: 48077639c94e2e02c1a572b4f7a33fc1b80c418149f094115e99b5fd424638d6 2024-04-25T14:59:33.673 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 14:59:33 smithi094 systemd[1]: Started Ceph mds.g for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:33.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.885+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5dc062b50 con 0x7ff5dc105d80 2024-04-25T14:59:33.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.886+0000 7ff5d3e00640 1 -- 172.21.15.94:0/1121883991 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7ff5d409cc10 con 0x7ff5dc105d80 2024-04-25T14:59:33.887 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:33.888 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-25T14:59:33.888 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 11 2024-04-25T14:59:33.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.888+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff5c00a3c30 msgr2=0x7ff5c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:33.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.888+0000 7ff5e2600640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff5c00a3c30 0x7ff5c00a60f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc005ab0 tx=0x7ff5cc005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:33.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.888+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc105d80 msgr2=0x7ff5dc1a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.888+0000 7ff5e2600640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc105d80 0x7ff5dc1a19f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400eb10 tx=0x7ff5d400c2d0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 shutdown_connections 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff5c00a3c30 0x7ff5c00a60f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5dc105d80 0x7ff5dc1a19f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5dc1053c0 0x7ff5dc19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 --2- 172.21.15.94:0/1121883991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5dc1041d0 0x7ff5dc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 >> 172.21.15.94:0/1121883991 conn(0x7ff5dc0ff740 msgr2=0x7ff5dc108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 shutdown_connections 2024-04-25T14:59:33.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:33.889+0000 7ff5e2600640 1 -- 172.21.15.94:0/1121883991 wait complete. 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.94:6838/3586316280,v1:172.21.15.94:6839/3586316280] up:boot 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: fsmap 10 up:standby 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:34.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:34.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: Deploying daemon mds.a on smithi094 2024-04-25T14:59:34.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:34 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1121883991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.94:6838/3586316280,v1:172.21.15.94:6839/3586316280] up:boot 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: fsmap 10 up:standby 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:34.781 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: Deploying daemon mds.a on smithi094 2024-04-25T14:59:34.782 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:34 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1121883991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.94:6838/3586316280,v1:172.21.15.94:6839/3586316280] up:boot 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: fsmap 10 up:standby 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:34.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:34.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:34.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: Deploying daemon mds.a on smithi094 2024-04-25T14:59:34.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:34 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1121883991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:35.910 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:36.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.330+0000 7f970ae00640 1 -- 172.21.15.94:0/2138983643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9704076910 msgr2=0x7f9704076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:36.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.330+0000 7f970ae00640 1 --2- 172.21.15.94:0/2138983643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9704076910 0x7f9704076d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009900 tx=0x7f96f402a400 comp rx=0 tx=0).stop 2024-04-25T14:59:36.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.330+0000 7f970ae00640 1 -- 172.21.15.94:0/2138983643 shutdown_connections 2024-04-25T14:59:36.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.330+0000 7f970ae00640 1 --2- 172.21.15.94:0/2138983643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97040784f0 0x7f9704078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.330+0000 7f970ae00640 1 --2- 172.21.15.94:0/2138983643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9704077b50 0x7f9704077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.330+0000 7f970ae00640 1 --2- 172.21.15.94:0/2138983643 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9704076910 0x7f9704076d30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.330+0000 7f970ae00640 1 -- 172.21.15.94:0/2138983643 >> 172.21.15.94:0/2138983643 conn(0x7f97040ff740 msgr2=0x7f9704101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.331+0000 7f970ae00640 1 -- 172.21.15.94:0/2138983643 shutdown_connections 2024-04-25T14:59:36.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.331+0000 7f970ae00640 1 -- 172.21.15.94:0/2138983643 wait complete. 2024-04-25T14:59:36.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.332+0000 7f970ae00640 1 Processor -- start 2024-04-25T14:59:36.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.333+0000 7f970ae00640 1 -- start start 2024-04-25T14:59:36.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.334+0000 7f970ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9704076910 0x7f97041045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:36.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.334+0000 7f970ae00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9704077b50 0x7f9704104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:36.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.334+0000 7f970ae00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97040784f0 0x7f970410bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.334+0000 7f970ae00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9704114110 con 0x7f9704077b50 2024-04-25T14:59:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.334+0000 7f970ae00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9704113f90 con 0x7f97040784f0 2024-04-25T14:59:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.334+0000 7f970ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9704114290 con 0x7f9704076910 2024-04-25T14:59:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.335+0000 7f9703e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9704076910 0x7f97041045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.335+0000 7f9703e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9704076910 0x7f97041045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43378/0 (socket says 172.21.15.94:43378) 2024-04-25T14:59:36.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.335+0000 7f9708e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97040784f0 0x7f970410bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.335+0000 7f9703e00640 1 -- 172.21.15.94:0/2588338411 learned_addr learned my addr 172.21.15.94:0/2588338411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.335+0000 7f9708e00640 1 -- 172.21.15.94:0/2588338411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9704076910 msgr2=0x7f97041045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.335+0000 7f9708e00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9704076910 0x7f97041045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.335+0000 7f9708e00640 1 -- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9704077b50 msgr2=0x7f9704104af0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T14:59:36.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.336+0000 7f9703400640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9704077b50 0x7f9704104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:36.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.336+0000 7f9708e00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9704077b50 0x7f9704104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.336+0000 7f9708e00640 1 -- 172.21.15.94:0/2588338411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970410c1d0 con 0x7f97040784f0 2024-04-25T14:59:36.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.336+0000 7f9703400640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9704077b50 0x7f9704104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T14:59:36.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.337+0000 7f9708e00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97040784f0 0x7f970410bb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009fd0 tx=0x7f96fc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:36.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.337+0000 7f9700c00640 1 -- 172.21.15.94:0/2588338411 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00ce20 con 0x7f97040784f0 2024-04-25T14:59:36.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.337+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97041aef00 con 0x7f97040784f0 2024-04-25T14:59:36.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.337+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97041af3c0 con 0x7f97040784f0 2024-04-25T14:59:36.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.337+0000 7f9700c00640 1 -- 172.21.15.94:0/2588338411 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96fc005020 con 0x7f97040784f0 2024-04-25T14:59:36.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.338+0000 7f9700c00640 1 -- 172.21.15.94:0/2588338411 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc010690 con 0x7f97040784f0 2024-04-25T14:59:36.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.340+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e40052c0 con 0x7f97040784f0 2024-04-25T14:59:36.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.341+0000 7f9700c00640 1 -- 172.21.15.94:0/2588338411 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f96fc004a40 con 0x7f97040784f0 2024-04-25T14:59:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.343+0000 7f9700c00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f96e80a3c80 0x7f96e80a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.343+0000 7f9700c00640 1 -- 172.21.15.94:0/2588338411 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f96fc0d0560 con 0x7f97040784f0 2024-04-25T14:59:36.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.343+0000 7f9703e00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f96e80a3c80 0x7f96e80a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:36.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.344+0000 7f9703e00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f96e80a3c80 0x7f96e80a6140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f96f40096c0 tx=0x7f96f40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:36.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.347+0000 7f9700c00640 1 -- 172.21.15.94:0/2588338411 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96fc09d340 con 0x7f97040784f0 2024-04-25T14:59:36.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.595+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96e40055b0 con 0x7f97040784f0 2024-04-25T14:59:36.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.596+0000 7f9700c00640 1 -- 172.21.15.94:0/2588338411 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8839 (secure 0 0 0) 0x7f96fc09cc40 con 0x7f97040784f0 2024-04-25T14:59:36.597 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:36.597 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-25T14:59:36.598 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 11 2024-04-25T14:59:36.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f96e80a3c80 msgr2=0x7f96e80a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:36.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f96e80a3c80 0x7f96e80a6140 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f96f40096c0 tx=0x7f96f40314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97040784f0 msgr2=0x7f970410bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97040784f0 0x7f970410bb60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009fd0 tx=0x7f96fc00ef10 comp rx=0 tx=0).stop 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 shutdown_connections 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f96e80a3c80 0x7f96e80a6140 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f97040784f0 0x7f970410bb60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9704077b50 0x7f9704104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 --2- 172.21.15.94:0/2588338411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9704076910 0x7f97041045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.599+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 >> 172.21.15.94:0/2588338411 conn(0x7f97040ff740 msgr2=0x7f9704101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.600+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 shutdown_connections 2024-04-25T14:59:36.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:36.600+0000 7f970ae00640 1 -- 172.21.15.94:0/2588338411 wait complete. 2024-04-25T14:59:36.760 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:36 smithi138 ceph-mon[32875]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:36.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:36 smithi094 ceph-mon[27862]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:36 smithi137 ceph-mon[32439]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:37.193 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:59:36 smithi094 podman[69510]: 2024-04-25 14:59:36.930139618 +0000 UTC m=+0.131466397 container create 8c712da2151967d7ce5986e3403b63c5fadc6b5d47842ee0992fd574334bd437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:59:37.194 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:59:37 smithi094 podman[69510]: 2024-04-25 14:59:37.192007668 +0000 UTC m=+0.393334444 container init 8c712da2151967d7ce5986e3403b63c5fadc6b5d47842ee0992fd574334bd437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T14:59:37.463 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:59:37 smithi094 podman[69510]: 2024-04-25 14:59:37.20336919 +0000 UTC m=+0.404695959 container start 8c712da2151967d7ce5986e3403b63c5fadc6b5d47842ee0992fd574334bd437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T14:59:37.463 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-mds[69529]: -- 172.21.15.94:0/2991669071 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cdabb51a40 con 0x55cdabb24800 2024-04-25T14:59:37.463 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a[69525]: starting mds.a at 2024-04-25T14:59:37.463 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:59:37 smithi094 bash[69510]: 8c712da2151967d7ce5986e3403b63c5fadc6b5d47842ee0992fd574334bd437 2024-04-25T14:59:37.463 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 14:59:37 smithi094 systemd[1]: Started Ceph mds.a for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:37 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2588338411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:37.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:37 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:37.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2588338411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:37.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:37.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:37 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:37 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2588338411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:37.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T14:59:37.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:37 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T14:59:38.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:38.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:38 smithi137 ceph-mon[32439]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:38.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:38.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:38 smithi137 ceph-mon[32439]: Deploying daemon mds.b on smithi137 2024-04-25T14:59:38.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:38 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.94:6840/2559487009,v1:172.21.15.94:6841/2559487009] up:boot 2024-04-25T14:59:38.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:38 smithi137 ceph-mon[32439]: fsmap 11 up:standby 2024-04-25T14:59:38.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:38 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:59:38.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 -- 172.21.15.94:0/3093034504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b941019c0 msgr2=0x7f5b94101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:38.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 --2- 172.21.15.94:0/3093034504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b941019c0 0x7f5b94101e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009ca0 tx=0x7f5b8c02a950 comp rx=0 tx=0).stop 2024-04-25T14:59:38.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 -- 172.21.15.94:0/3093034504 shutdown_connections 2024-04-25T14:59:38.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 --2- 172.21.15.94:0/3093034504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b941019c0 0x7f5b94101e40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 --2- 172.21.15.94:0/3093034504 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b94101020 0x7f5b94101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 --2- 172.21.15.94:0/3093034504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 0x7f5b94100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 -- 172.21.15.94:0/3093034504 >> 172.21.15.94:0/3093034504 conn(0x7f5b940fc220 msgr2=0x7f5b940fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 -- 172.21.15.94:0/3093034504 shutdown_connections 2024-04-25T14:59:38.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.836+0000 7f5b9b000640 1 -- 172.21.15.94:0/3093034504 wait complete. 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.837+0000 7f5b9b000640 1 Processor -- start 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.837+0000 7f5b9b000640 1 -- start start 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b9b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 0x7f5b9419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b9b000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b94101020 0x7f5b9419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b9b000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b941019c0 0x7f5b941a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b9b000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410fb60 con 0x7f5b94101020 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b9b000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410f9e0 con 0x7f5b941019c0 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b9b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9410fce0 con 0x7f5b941006c0 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 0x7f5b9419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b94101020 0x7f5b9419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 0x7f5b9419a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:45236/0 (socket says 172.21.15.94:45236) 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99c00640 1 -- 172.21.15.94:0/1382155375 learned_addr learned my addr 172.21.15.94:0/1382155375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b9a600640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b941019c0 0x7f5b941a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99c00640 1 -- 172.21.15.94:0/1382155375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b941019c0 msgr2=0x7f5b941a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99c00640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b941019c0 0x7f5b941a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99c00640 1 -- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b94101020 msgr2=0x7f5b9419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.838+0000 7f5b99c00640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b94101020 0x7f5b9419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b99c00640 1 -- 172.21.15.94:0/1382155375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b941a1dd0 con 0x7f5b941006c0 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b99200640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b94101020 0x7f5b9419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b99c00640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 0x7f5b9419a1b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e9f0 tx=0x7f5b8800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b92000640 1 -- 172.21.15.94:0/1382155375 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800cd80 con 0x7f5b941006c0 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b92000640 1 -- 172.21.15.94:0/1382155375 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b880054d0 con 0x7f5b941006c0 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b941a20c0 con 0x7f5b941006c0 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b92000640 1 -- 172.21.15.94:0/1382155375 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88010690 con 0x7f5b941006c0 2024-04-25T14:59:38.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.839+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b941a25c0 con 0x7f5b941006c0 2024-04-25T14:59:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.840+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9410b410 con 0x7f5b941006c0 2024-04-25T14:59:38.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.842+0000 7f5b92000640 1 -- 172.21.15.94:0/1382155375 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b88004e90 con 0x7f5b941006c0 2024-04-25T14:59:38.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.844+0000 7f5b92000640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b7c0a3bb0 0x7f5b7c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:38.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.844+0000 7f5b92000640 1 -- 172.21.15.94:0/1382155375 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5b880d03f0 con 0x7f5b941006c0 2024-04-25T14:59:38.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.844+0000 7f5b99200640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b7c0a3bb0 0x7f5b7c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:38.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.845+0000 7f5b99200640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b7c0a3bb0 0x7f5b7c0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b9419b6c0 tx=0x7f5b80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:38.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:38.848+0000 7f5b92000640 1 -- 172.21.15.94:0/1382155375 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b8809d1d0 con 0x7f5b941006c0 2024-04-25T14:59:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:38 smithi138 ceph-mon[32875]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:38 smithi138 ceph-mon[32875]: Deploying daemon mds.b on smithi137 2024-04-25T14:59:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:38 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.94:6840/2559487009,v1:172.21.15.94:6841/2559487009] up:boot 2024-04-25T14:59:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:38 smithi138 ceph-mon[32875]: fsmap 11 up:standby 2024-04-25T14:59:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:38 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:59:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:38 smithi094 ceph-mon[27862]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:38 smithi094 ceph-mon[27862]: Deploying daemon mds.b on smithi137 2024-04-25T14:59:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:38 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.94:6840/2559487009,v1:172.21.15.94:6841/2559487009] up:boot 2024-04-25T14:59:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:38 smithi094 ceph-mon[27862]: fsmap 11 up:standby 2024-04-25T14:59:38.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:38 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T14:59:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.090+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b9402d5c0 con 0x7f5b941006c0 2024-04-25T14:59:39.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.091+0000 7f5b92000640 1 -- 172.21.15.94:0/1382155375 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9665 (secure 0 0 0) 0x7f5b8809cad0 con 0x7f5b941006c0 2024-04-25T14:59:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:39.092 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-25T14:59:39.093 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-04-25T14:59:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b7c0a3bb0 msgr2=0x7f5b7c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b7c0a3bb0 0x7f5b7c0a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b9419b6c0 tx=0x7f5b80005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 msgr2=0x7f5b9419a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:39.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 0x7f5b9419a1b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e9f0 tx=0x7f5b8800edf0 comp rx=0 tx=0).stop 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 shutdown_connections 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b7c0a3bb0 0x7f5b7c0a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b941019c0 0x7f5b941a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b94101020 0x7f5b9419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 --2- 172.21.15.94:0/1382155375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b941006c0 0x7f5b9419a1b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 >> 172.21.15.94:0/1382155375 conn(0x7f5b940fc220 msgr2=0x7f5b940fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 shutdown_connections 2024-04-25T14:59:39.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:39.095+0000 7f5b9b000640 1 -- 172.21.15.94:0/1382155375 wait complete. 2024-04-25T14:59:39.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:39 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1382155375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:39.733 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:39 smithi137 systemd[1]: Starting Ceph mds.b for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T14:59:39.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:39 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1382155375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:39 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1382155375' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:40.117 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:39 smithi137 podman[63277]: 2024-04-25 14:59:39.758198836 +0000 UTC m=+0.018078872 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T14:59:40.117 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:39 smithi137 podman[63277]: 2024-04-25 14:59:39.872503096 +0000 UTC m=+0.132383133 container create 0c3ed092aa3c85e66e143cf05f30c0e23500f04b8bf87524f4c8d32de774e357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T14:59:40.117 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:40 smithi137 podman[63277]: 2024-04-25 14:59:40.091538217 +0000 UTC m=+0.351418251 container init 0c3ed092aa3c85e66e143cf05f30c0e23500f04b8bf87524f4c8d32de774e357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T14:59:40.117 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:40 smithi137 podman[63277]: 2024-04-25 14:59:40.094197305 +0000 UTC m=+0.354077344 container start 0c3ed092aa3c85e66e143cf05f30c0e23500f04b8bf87524f4c8d32de774e357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T14:59:40.482 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-mds[63296]: -- 172.21.15.137:0/3735698037 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef02f89a40 con 0x55ef02f5cc00 2024-04-25T14:59:40.482 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b[63292]: starting mds.b at 2024-04-25T14:59:40.482 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:40 smithi137 bash[63277]: 0c3ed092aa3c85e66e143cf05f30c0e23500f04b8bf87524f4c8d32de774e357 2024-04-25T14:59:40.482 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:40 smithi137 systemd[1]: Started Ceph mds.b for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T14:59:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:40 smithi094 ceph-mon[27862]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.865 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:40 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:40 smithi138 ceph-mon[32875]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:40 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-mon[32439]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:40 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:41.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.309+0000 7ff6d6a00640 1 -- 172.21.15.94:0/915662565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 msgr2=0x7ff6d0105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.309+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/915662565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d0105820 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc009900 tx=0x7ff6bc02a410 comp rx=0 tx=0).stop 2024-04-25T14:59:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.310+0000 7ff6d6a00640 1 -- 172.21.15.94:0/915662565 shutdown_connections 2024-04-25T14:59:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.310+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/915662565 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d0105d60 0x7ff6d010c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.310+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/915662565 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d0105820 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.310+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/915662565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6d01041b0 0x7ff6d01045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.310+0000 7ff6d6a00640 1 -- 172.21.15.94:0/915662565 >> 172.21.15.94:0/915662565 conn(0x7ff6d00ff740 msgr2=0x7ff6d0101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.310+0000 7ff6d6a00640 1 -- 172.21.15.94:0/915662565 shutdown_connections 2024-04-25T14:59:41.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.310+0000 7ff6d6a00640 1 -- 172.21.15.94:0/915662565 wait complete. 2024-04-25T14:59:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 Processor -- start 2024-04-25T14:59:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 -- start start 2024-04-25T14:59:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01041b0 0x7ff6d019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6d0105d60 0x7ff6d01a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010fcf0 con 0x7ff6d0105d60 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010fb70 con 0x7ff6d01041b0 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.311+0000 7ff6d6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d010fe70 con 0x7ff6d01053a0 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6d4a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6d0105d60 0x7ff6d01a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:45246/0 (socket says 172.21.15.94:45246) 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 -- 172.21.15.94:0/3928001986 learned_addr learned my addr 172.21.15.94:0/3928001986 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cfe00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01041b0 0x7ff6d019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 -- 172.21.15.94:0/3928001986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01041b0 msgr2=0x7ff6d019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01041b0 0x7ff6d019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 -- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6d0105d60 msgr2=0x7ff6d01a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6d0105d60 0x7ff6d01a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 -- 172.21.15.94:0/3928001986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d01a2000 con 0x7ff6d01053a0 2024-04-25T14:59:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.312+0000 7ff6cf400640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d019a920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc005940 tx=0x7ff6bc031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.313+0000 7ff6ccc00640 1 -- 172.21.15.94:0/3928001986 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc03f030 con 0x7ff6d01053a0 2024-04-25T14:59:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.313+0000 7ff6ccc00640 1 -- 172.21.15.94:0/3928001986 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6bc003aa0 con 0x7ff6d01053a0 2024-04-25T14:59:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.313+0000 7ff6ccc00640 1 -- 172.21.15.94:0/3928001986 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc035070 con 0x7ff6d01053a0 2024-04-25T14:59:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.313+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d01a2290 con 0x7ff6d01053a0 2024-04-25T14:59:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.313+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d01a27d0 con 0x7ff6d01053a0 2024-04-25T14:59:41.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.314+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d010b4f0 con 0x7ff6d01053a0 2024-04-25T14:59:41.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.316+0000 7ff6ccc00640 1 -- 172.21.15.94:0/3928001986 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff6bc042030 con 0x7ff6d01053a0 2024-04-25T14:59:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.317+0000 7ff6ccc00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6b40a3c30 0x7ff6b40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.318+0000 7ff6ccc00640 1 -- 172.21.15.94:0/3928001986 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff6bc0f08c0 con 0x7ff6d01053a0 2024-04-25T14:59:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.318+0000 7ff6cfe00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6b40a3c30 0x7ff6b40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.318+0000 7ff6cfe00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6b40a3c30 0x7ff6b40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0105200 tx=0x7ff6c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:41.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.319+0000 7ff6ccc00640 1 -- 172.21.15.94:0/3928001986 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6bc0bd050 con 0x7ff6d01053a0 2024-04-25T14:59:41.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.582+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6d002d5c0 con 0x7ff6d01053a0 2024-04-25T14:59:41.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.583+0000 7ff6ccc00640 1 -- 172.21.15.94:0/3928001986 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10494 (secure 0 0 0) 0x7ff6bc0c1350 con 0x7ff6d01053a0 2024-04-25T14:59:41.583 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:41.583 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24596,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T14:59:41.583 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 13 2024-04-25T14:59:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6b40a3c30 msgr2=0x7ff6b40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6b40a3c30 0x7ff6b40a60f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0105200 tx=0x7ff6c0005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 msgr2=0x7ff6d019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d019a920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc005940 tx=0x7ff6bc031a50 comp rx=0 tx=0).stop 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 shutdown_connections 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6b40a3c30 0x7ff6b40a60f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6d0105d60 0x7ff6d01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d01053a0 0x7ff6d019a920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 --2- 172.21.15.94:0/3928001986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6d01041b0 0x7ff6d019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 >> 172.21.15.94:0/3928001986 conn(0x7ff6d00ff740 msgr2=0x7ff6d01086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 shutdown_connections 2024-04-25T14:59:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.586+0000 7ff6d6a00640 1 -- 172.21.15.94:0/3928001986 wait complete. 2024-04-25T14:59:41.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:59:41.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:41 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] up:boot 2024-04-25T14:59:41.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:41 smithi137 ceph-mon[32439]: fsmap 12 up:standby 2024-04-25T14:59:41.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:59:41.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:41.663 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:41 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:41 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] up:boot 2024-04-25T14:59:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:41 smithi138 ceph-mon[32875]: fsmap 12 up:standby 2024-04-25T14:59:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:59:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:41 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:41.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 -- 172.21.15.94:0/4288842348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c101fa0 msgr2=0x7ffb7c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 --2- 172.21.15.94:0/4288842348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c1023c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009900 tx=0x7ffb7002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 -- 172.21.15.94:0/4288842348 shutdown_connections 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 --2- 172.21.15.94:0/4288842348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c103b50 0x7ffb7c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 --2- 172.21.15.94:0/4288842348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb7c103190 0x7ffb7c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 --2- 172.21.15.94:0/4288842348 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c1023c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 -- 172.21.15.94:0/4288842348 >> 172.21.15.94:0/4288842348 conn(0x7ffb7c0fd530 msgr2=0x7ffb7c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 -- 172.21.15.94:0/4288842348 shutdown_connections 2024-04-25T14:59:41.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.784+0000 7ffb83e00640 1 -- 172.21.15.94:0/4288842348 wait complete. 2024-04-25T14:59:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 Processor -- start 2024-04-25T14:59:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 -- start start 2024-04-25T14:59:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c103190 0x7ffb7c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb7c103b50 0x7ffb7c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10db30 con 0x7ffb7c101fa0 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10d9b0 con 0x7ffb7c103190 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.785+0000 7ffb83e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c10dcb0 con 0x7ffb7c103b50 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c103190 0x7ffb7c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb81400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb81400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54968/0 (socket says 172.21.15.94:54968) 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c103190 0x7ffb7c19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:44698/0 (socket says 172.21.15.94:44698) 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb81400640 1 -- 172.21.15.94:0/1868660795 learned_addr learned my addr 172.21.15.94:0/1868660795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb81e00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb7c103b50 0x7ffb7c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 -- 172.21.15.94:0/1868660795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb7c103b50 msgr2=0x7ffb7c1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb7c103b50 0x7ffb7c1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 -- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c101fa0 msgr2=0x7ffb7c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 -- 172.21.15.94:0/1868660795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c1a1ff0 con 0x7ffb7c103190 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb81400640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.786+0000 7ffb80a00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c103190 0x7ffb7c19a910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800d720 tx=0x7ffb6800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.787+0000 7ffb7a000640 1 -- 172.21.15.94:0/1868660795 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb68017070 con 0x7ffb7c103190 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.787+0000 7ffb7a000640 1 -- 172.21.15.94:0/1868660795 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb6800de30 con 0x7ffb7c103190 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.787+0000 7ffb7a000640 1 -- 172.21.15.94:0/1868660795 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb680047a0 con 0x7ffb7c103190 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.787+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c1a22e0 con 0x7ffb7c103190 2024-04-25T14:59:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.787+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c1a27e0 con 0x7ffb7c103190 2024-04-25T14:59:41.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.788+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c1a2a40 con 0x7ffb7c103190 2024-04-25T14:59:41.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.790+0000 7ffb7a000640 1 -- 172.21.15.94:0/1868660795 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffb680099d0 con 0x7ffb7c103190 2024-04-25T14:59:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.792+0000 7ffb7a000640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffb640a3c30 0x7ffb640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.792+0000 7ffb7a000640 1 -- 172.21.15.94:0/1868660795 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffb680cfb80 con 0x7ffb7c103190 2024-04-25T14:59:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.792+0000 7ffb81400640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffb640a3c30 0x7ffb640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.793+0000 7ffb81400640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffb640a3c30 0x7ffb640a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000c00 tx=0x7ffb70005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:41.796+0000 7ffb7a000640 1 -- 172.21.15.94:0/1868660795 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb68020020 con 0x7ffb7c103190 2024-04-25T14:59:41.827 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:41 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] up:boot 2024-04-25T14:59:41.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:41 smithi094 ceph-mon[27862]: fsmap 12 up:standby 2024-04-25T14:59:41.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T14:59:41.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:41.828 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:41 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.005+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb7c1a2d30 con 0x7ffb7c103190 2024-04-25T14:59:42.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.006+0000 7ffb7a000640 1 -- 172.21.15.94:0/1868660795 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18441 (secure 0 0 0) 0x7ffb6809c960 con 0x7ffb7c103190 2024-04-25T14:59:42.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:42.007 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":82,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T14:58:52.443830+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffb640a3c30 msgr2=0x7ffb640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffb640a3c30 0x7ffb640a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffb70000c00 tx=0x7ffb70005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c103190 msgr2=0x7ffb7c19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c103190 0x7ffb7c19a910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800d720 tx=0x7ffb6800dbe0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 shutdown_connections 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffb640a3c30 0x7ffb640a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb7c103b50 0x7ffb7c1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffb7c103190 0x7ffb7c19a910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 --2- 172.21.15.94:0/1868660795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb7c101fa0 0x7ffb7c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 >> 172.21.15.94:0/1868660795 conn(0x7ffb7c0fd530 msgr2=0x7ffb7c074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:42.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 shutdown_connections 2024-04-25T14:59:42.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.009+0000 7ffb83e00640 1 -- 172.21.15.94:0/1868660795 wait complete. 2024-04-25T14:59:42.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T14:59:42.028 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T14:59:42.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 -- 172.21.15.94:0/2754665335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 msgr2=0x7f2780109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 --2- 172.21.15.94:0/2754665335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 0x7f2780109c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2768009860 tx=0x7f276802a340 comp rx=0 tx=0).stop 2024-04-25T14:59:42.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 -- 172.21.15.94:0/2754665335 shutdown_connections 2024-04-25T14:59:42.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 --2- 172.21.15.94:0/2754665335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278010a160 0x7f27801109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 --2- 172.21.15.94:0/2754665335 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 0x7f2780109c20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 --2- 172.21.15.94:0/2754665335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 0x7f27801089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 -- 172.21.15.94:0/2754665335 >> 172.21.15.94:0/2754665335 conn(0x7f2780077550 msgr2=0x7f2780079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:42.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 -- 172.21.15.94:0/2754665335 shutdown_connections 2024-04-25T14:59:42.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.219+0000 7f2787c00640 1 -- 172.21.15.94:0/2754665335 wait complete. 2024-04-25T14:59:42.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 Processor -- start 2024-04-25T14:59:42.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 -- start start 2024-04-25T14:59:42.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 0x7f278019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 0x7f278019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278010a160 0x7f27801a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f27801140e0 con 0x7f27801085b0 2024-04-25T14:59:42.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2780113f60 con 0x7f27801097a0 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2787c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2780114260 con 0x7f278010a160 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2785200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 0x7f278019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2785200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 0x7f278019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54992/0 (socket says 172.21.15.94:54992) 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2785200640 1 -- 172.21.15.94:0/1509524005 learned_addr learned my addr 172.21.15.94:0/1509524005 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2785c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278010a160 0x7f27801a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.221+0000 7f2777e00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 0x7f278019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2785c00640 1 -- 172.21.15.94:0/1509524005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 msgr2=0x7f278019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2785c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 0x7f278019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2785c00640 1 -- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 msgr2=0x7f278019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2785c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 0x7f278019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2785c00640 1 -- 172.21.15.94:0/1509524005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27801a6400 con 0x7f278010a160 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2785200640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 0x7f278019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:42.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2785c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278010a160 0x7f27801a5d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2778009fd0 tx=0x7f277800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:42.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f277e000640 1 -- 172.21.15.94:0/1509524005 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277800ce20 con 0x7f278010a160 2024-04-25T14:59:42.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f277e000640 1 -- 172.21.15.94:0/1509524005 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2778005020 con 0x7f278010a160 2024-04-25T14:59:42.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27801a66f0 con 0x7f278010a160 2024-04-25T14:59:42.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.222+0000 7f277e000640 1 -- 172.21.15.94:0/1509524005 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778010690 con 0x7f278010a160 2024-04-25T14:59:42.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.223+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27801a6c60 con 0x7f278010a160 2024-04-25T14:59:42.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.224+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278010f970 con 0x7f278010a160 2024-04-25T14:59:42.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.225+0000 7f277e000640 1 -- 172.21.15.94:0/1509524005 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f27780054d0 con 0x7f278010a160 2024-04-25T14:59:42.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.226+0000 7f277e000640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f27640a3c30 0x7f27640a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.227+0000 7f277e000640 1 -- 172.21.15.94:0/1509524005 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f27780d0640 con 0x7f278010a160 2024-04-25T14:59:42.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.227+0000 7f2785200640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f27640a3c30 0x7f27640a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.227+0000 7f2785200640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f27640a3c30 0x7f27640a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2770011490 tx=0x7f2770009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:42.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.230+0000 7f277e000640 1 -- 172.21.15.94:0/1509524005 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f277809d420 con 0x7f278010a160 2024-04-25T14:59:42.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.450+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f278002d5c0 con 0x7f278010a160 2024-04-25T14:59:42.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.451+0000 7f277e000640 1 -- 172.21.15.94:0/1509524005 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f277809cd20 con 0x7f278010a160 2024-04-25T14:59:42.452 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T14:59:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.453+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f27640a3c30 msgr2=0x7f27640a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.453+0000 7f2787c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f27640a3c30 0x7f27640a60f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2770011490 tx=0x7f2770009260 comp rx=0 tx=0).stop 2024-04-25T14:59:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.453+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278010a160 msgr2=0x7f27801a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.453+0000 7f2787c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278010a160 0x7f27801a5d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2778009fd0 tx=0x7f277800ef10 comp rx=0 tx=0).stop 2024-04-25T14:59:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 shutdown_connections 2024-04-25T14:59:42.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f27640a3c30 0x7f27640a60f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278010a160 0x7f27801a5d90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f27801097a0 0x7f278019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 --2- 172.21.15.94:0/1509524005 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27801085b0 0x7f278019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 >> 172.21.15.94:0/1509524005 conn(0x7f2780077550 msgr2=0x7f278010cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 shutdown_connections 2024-04-25T14:59:42.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.454+0000 7f2787c00640 1 -- 172.21.15.94:0/1509524005 wait complete. 2024-04-25T14:59:42.474 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T14:59:42.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 -- 172.21.15.94:0/554919582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe741041b0 msgr2=0x7fbe741045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 --2- 172.21.15.94:0/554919582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe741041b0 0x7fbe741045d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009900 tx=0x7fbe6402a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 -- 172.21.15.94:0/554919582 shutdown_connections 2024-04-25T14:59:42.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 --2- 172.21.15.94:0/554919582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe74105d60 0x7fbe7410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 --2- 172.21.15.94:0/554919582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741053a0 0x7fbe74105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 --2- 172.21.15.94:0/554919582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe741041b0 0x7fbe741045d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 -- 172.21.15.94:0/554919582 >> 172.21.15.94:0/554919582 conn(0x7fbe740ff740 msgr2=0x7fbe74101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:42.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 -- 172.21.15.94:0/554919582 shutdown_connections 2024-04-25T14:59:42.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.657+0000 7fbe7a000640 1 -- 172.21.15.94:0/554919582 wait complete. 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.658+0000 7fbe7a000640 1 Processor -- start 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe7a000640 1 -- start start 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe7a000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741041b0 0x7fbe7419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe7a000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe741053a0 0x7fbe7419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe7a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe74105d60 0x7fbe741a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe7a000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7410fd10 con 0x7fbe741053a0 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe7a000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7410fb90 con 0x7fbe741041b0 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe7a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7410fe90 con 0x7fbe74105d60 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe74105d60 0x7fbe741a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe72a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe741053a0 0x7fbe7419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe73400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741041b0 0x7fbe7419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe74105d60 0x7fbe741a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:45302/0 (socket says 172.21.15.94:45302) 2024-04-25T14:59:42.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe73400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741041b0 0x7fbe7419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:44754/0 (socket says 172.21.15.94:44754) 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.659+0000 7fbe73e00640 1 -- 172.21.15.94:0/2181438003 learned_addr learned my addr 172.21.15.94:0/2181438003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe73e00640 1 -- 172.21.15.94:0/2181438003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741041b0 msgr2=0x7fbe7419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe73e00640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741041b0 0x7fbe7419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe73e00640 1 -- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe741053a0 msgr2=0x7fbe7419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe73e00640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe741053a0 0x7fbe7419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe73e00640 1 -- 172.21.15.94:0/2181438003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe741a2060 con 0x7fbe74105d60 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe73400640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741041b0 0x7fbe7419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe73e00640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe74105d60 0x7fbe741a19f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00efd0 tx=0x7fbe6c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe6be00640 1 -- 172.21.15.94:0/2181438003 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c003b70 con 0x7fbe74105d60 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe6be00640 1 -- 172.21.15.94:0/2181438003 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe6c003d10 con 0x7fbe74105d60 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe741a2350 con 0x7fbe74105d60 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe6be00640 1 -- 172.21.15.94:0/2181438003 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0108a0 con 0x7fbe74105d60 2024-04-25T14:59:42.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.660+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe741a2850 con 0x7fbe74105d60 2024-04-25T14:59:42.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.662+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7410b540 con 0x7fbe74105d60 2024-04-25T14:59:42.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.663+0000 7fbe6be00640 1 -- 172.21.15.94:0/2181438003 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbe6c004020 con 0x7fbe74105d60 2024-04-25T14:59:42.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.665+0000 7fbe6be00640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe580a3bb0 0x7fbe580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:42.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.665+0000 7fbe6be00640 1 -- 172.21.15.94:0/2181438003 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbe6c0d03d0 con 0x7fbe74105d60 2024-04-25T14:59:42.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.666+0000 7fbe73400640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe580a3bb0 0x7fbe580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:42.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.666+0000 7fbe73400640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe580a3bb0 0x7fbe580a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005ab0 tx=0x7fbe64005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:42.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.668+0000 7fbe6be00640 1 -- 172.21.15.94:0/2181438003 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe6c09d1b0 con 0x7fbe74105d60 2024-04-25T14:59:42.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.896+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbe74062b50 con 0x7fbe74105d60 2024-04-25T14:59:42.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.896+0000 7fbe6be00640 1 -- 172.21.15.94:0/2181438003 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fbe6c09cab0 con 0x7fbe74105d60 2024-04-25T14:59:42.897 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:42.897 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-04-25T14:59:42.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T14:59:42.897 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-04-25T14:59:42.897 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-04-25T14:59:42.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T14:59:42.898 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-04-25T14:59:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe580a3bb0 msgr2=0x7fbe580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe580a3bb0 0x7fbe580a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005ab0 tx=0x7fbe64005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe74105d60 msgr2=0x7fbe741a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe74105d60 0x7fbe741a19f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00efd0 tx=0x7fbe6c00c6a0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 shutdown_connections 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe580a3bb0 0x7fbe580a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe74105d60 0x7fbe741a19f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe741053a0 0x7fbe7419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 --2- 172.21.15.94:0/2181438003 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe741041b0 0x7fbe7419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 >> 172.21.15.94:0/2181438003 conn(0x7fbe740ff740 msgr2=0x7fbe74108710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 shutdown_connections 2024-04-25T14:59:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:42.899+0000 7fbe7a000640 1 -- 172.21.15.94:0/2181438003 wait complete. 2024-04-25T14:59:42.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:42.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3928001986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:42.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1868660795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:42.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.915 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.916 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:42 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1509524005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:59:42.918 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T14:59:42.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3928001986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1868660795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:42 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1509524005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3928001986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1868660795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:42 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1509524005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T14:59:43.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.116+0000 7effffe00640 1 -- 172.21.15.94:0/882789255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0000103fb0 msgr2=0x7f00001043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:43.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.116+0000 7effffe00640 1 --2- 172.21.15.94:0/882789255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0000103fb0 0x7f00001043d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efff0009900 tx=0x7efff002a410 comp rx=0 tx=0).stop 2024-04-25T14:59:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.116+0000 7effffe00640 1 -- 172.21.15.94:0/882789255 shutdown_connections 2024-04-25T14:59:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.116+0000 7effffe00640 1 --2- 172.21.15.94:0/882789255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 0x7f000010c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.116+0000 7effffe00640 1 --2- 172.21.15.94:0/882789255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00001051a0 0x7f0000105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.116+0000 7effffe00640 1 --2- 172.21.15.94:0/882789255 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0000103fb0 0x7f00001043d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.116+0000 7effffe00640 1 -- 172.21.15.94:0/882789255 >> 172.21.15.94:0/882789255 conn(0x7f00000ff520 msgr2=0x7f0000101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.117+0000 7effffe00640 1 -- 172.21.15.94:0/882789255 shutdown_connections 2024-04-25T14:59:43.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.117+0000 7effffe00640 1 -- 172.21.15.94:0/882789255 wait complete. 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 Processor -- start 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 -- start start 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0000103fb0 0x7f000019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00001051a0 0x7f000019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 0x7f00001a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f000010fb00 con 0x7f0000105b60 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f000010f980 con 0x7f00001051a0 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effffe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f000010fc80 con 0x7f0000103fb0 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effff400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 0x7f00001a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7efffe000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00001051a0 0x7f000019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effff400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 0x7f00001a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55028/0 (socket says 172.21.15.94:55028) 2024-04-25T14:59:43.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7effff400640 1 -- 172.21.15.94:0/310860445 learned_addr learned my addr 172.21.15.94:0/310860445 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.118+0000 7efffea00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0000103fb0 0x7f000019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7effff400640 1 -- 172.21.15.94:0/310860445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0000103fb0 msgr2=0x7f000019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7effff400640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0000103fb0 0x7f000019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7effff400640 1 -- 172.21.15.94:0/310860445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00001051a0 msgr2=0x7f000019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7effff400640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00001051a0 0x7f000019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7effff400640 1 -- 172.21.15.94:0/310860445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00001a1dd0 con 0x7f0000105b60 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7effff400640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 0x7f00001a1760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efff800eb10 tx=0x7efff800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7f0004a00640 1 -- 172.21.15.94:0/310860445 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8004b20 con 0x7f0000105b60 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7f0004a00640 1 -- 172.21.15.94:0/310860445 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff8004cc0 con 0x7f0000105b60 2024-04-25T14:59:43.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.119+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00001a20c0 con 0x7f0000105b60 2024-04-25T14:59:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.120+0000 7f0004a00640 1 -- 172.21.15.94:0/310860445 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8010670 con 0x7f0000105b60 2024-04-25T14:59:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.120+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00001a25c0 con 0x7f0000105b60 2024-04-25T14:59:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.121+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000010b350 con 0x7f0000105b60 2024-04-25T14:59:43.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.123+0000 7f0004a00640 1 -- 172.21.15.94:0/310860445 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efff8005000 con 0x7f0000105b60 2024-04-25T14:59:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.124+0000 7f0004a00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7effe40a3bb0 0x7effe40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:43.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.124+0000 7f0004a00640 1 -- 172.21.15.94:0/310860445 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efff80d04e0 con 0x7f0000105b60 2024-04-25T14:59:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.124+0000 7efffea00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7effe40a3bb0 0x7effe40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:43.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.125+0000 7efffea00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7effe40a3bb0 0x7effe40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff0005ab0 tx=0x7efff0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.129+0000 7f0004a00640 1 -- 172.21.15.94:0/310860445 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efff809d360 con 0x7f0000105b60 2024-04-25T14:59:43.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:43.344+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f000002d5c0 con 0x7f0000105b60 2024-04-25T14:59:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2181438003' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:59:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/310860445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:59:43.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:43 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:43.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2181438003' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:59:43.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:43.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:43.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/310860445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:59:43.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:43 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2181438003' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T14:59:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:43.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/310860445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T14:59:43.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:43 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T14:59:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.293+0000 7f0004a00640 1 -- 172.21.15.94:0/310860445 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7efff80bbe20 con 0x7f0000105b60 2024-04-25T14:59:44.294 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-04-25T14:59:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.297+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7effe40a3bb0 msgr2=0x7effe40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.297+0000 7effffe00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7effe40a3bb0 0x7effe40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efff0005ab0 tx=0x7efff0005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.297+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 msgr2=0x7f00001a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:44.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.297+0000 7effffe00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 0x7f00001a1760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efff800eb10 tx=0x7efff800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 shutdown_connections 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7effe40a3bb0 0x7effe40a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0000105b60 0x7f00001a1760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00001051a0 0x7f000019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 --2- 172.21.15.94:0/310860445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0000103fb0 0x7f000019a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 >> 172.21.15.94:0/310860445 conn(0x7f00000ff520 msgr2=0x7f0000108520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 shutdown_connections 2024-04-25T14:59:44.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.298+0000 7effffe00640 1 -- 172.21.15.94:0/310860445 wait complete. 2024-04-25T14:59:44.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T14:59:44.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 -- 172.21.15.94:0/2063229498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 msgr2=0x7f16340743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:44.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 --2- 172.21.15.94:0/2063229498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f16340743e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1624009900 tx=0x7f162402a410 comp rx=0 tx=0).stop 2024-04-25T14:59:44.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 -- 172.21.15.94:0/2063229498 shutdown_connections 2024-04-25T14:59:44.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 --2- 172.21.15.94:0/2063229498 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1634109d40 0x7f163410c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 --2- 172.21.15.94:0/2063229498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16340749b0 0x7f1634074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 --2- 172.21.15.94:0/2063229498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f16340743e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 -- 172.21.15.94:0/2063229498 >> 172.21.15.94:0/2063229498 conn(0x7f16340fd3f0 msgr2=0x7f16340ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:44.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 -- 172.21.15.94:0/2063229498 shutdown_connections 2024-04-25T14:59:44.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.515+0000 7f1633e00640 1 -- 172.21.15.94:0/2063229498 wait complete. 2024-04-25T14:59:44.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 Processor -- start 2024-04-25T14:59:44.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 -- start start 2024-04-25T14:59:44.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16340749b0 0x7f163419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:44.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f163419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1634109d40 0x7f16341a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f163410fc00 con 0x7f1634075f90 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f163410fa80 con 0x7f1634109d40 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.516+0000 7f1633e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f163410fd80 con 0x7f16340749b0 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f163419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f163419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55044/0 (socket says 172.21.15.94:55044) 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16340749b0 0x7f163419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 -- 172.21.15.94:0/2859955803 learned_addr learned my addr 172.21.15.94:0/2859955803 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1633400640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1634109d40 0x7f16341a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 -- 172.21.15.94:0/2859955803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16340749b0 msgr2=0x7f163419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16340749b0 0x7f163419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 -- 172.21.15.94:0/2859955803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1634109d40 msgr2=0x7f16341a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:44.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1634109d40 0x7f16341a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 -- 172.21.15.94:0/2859955803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16341a1f20 con 0x7f1634075f90 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1633400640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1634109d40 0x7f16341a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632a00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16340749b0 0x7f163419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.517+0000 7f1632000640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f163419a840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f162000e9f0 tx=0x7f162000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.518+0000 7f162b400640 1 -- 172.21.15.94:0/2859955803 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162000cd80 con 0x7f1634075f90 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.518+0000 7f162b400640 1 -- 172.21.15.94:0/2859955803 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16200054d0 con 0x7f1634075f90 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.518+0000 7f162b400640 1 -- 172.21.15.94:0/2859955803 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620010690 con 0x7f1634075f90 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.518+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16341a2210 con 0x7f1634075f90 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.518+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16341a2710 con 0x7f1634075f90 2024-04-25T14:59:44.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.519+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1634108a70 con 0x7f1634075f90 2024-04-25T14:59:44.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.522+0000 7f162b400640 1 -- 172.21.15.94:0/2859955803 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1620004e90 con 0x7f1634075f90 2024-04-25T14:59:44.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.523+0000 7f162b400640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f16180a3bb0 0x7f16180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:44.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.523+0000 7f162b400640 1 -- 172.21.15.94:0/2859955803 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f16200d0560 con 0x7f1634075f90 2024-04-25T14:59:44.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.524+0000 7f1632a00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f16180a3bb0 0x7f16180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:44.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.524+0000 7f1632a00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f16180a3bb0 0x7f16180a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1624031500 tx=0x7f1624005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:44.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.528+0000 7f162b400640 1 -- 172.21.15.94:0/2859955803 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f162009d1d0 con 0x7f1634075f90 2024-04-25T14:59:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:44 smithi138 ceph-mon[32875]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:44 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/310860445' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:59:44.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:44 smithi138 ceph-mon[32875]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T14:59:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:44 smithi094 ceph-mon[27862]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:44 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/310860445' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:59:44.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:44 smithi094 ceph-mon[27862]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T14:59:44.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:44.751+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f163402d5c0 con 0x7f1634075f90 2024-04-25T14:59:44.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:44 smithi137 ceph-mon[32439]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:44.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:44 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/310860445' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T14:59:44.828 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:44 smithi137 ceph-mon[32439]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T14:59:45.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.298+0000 7f162b400640 1 -- 172.21.15.94:0/2859955803 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f162009cad0 con 0x7f1634075f90 2024-04-25T14:59:45.299 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-04-25T14:59:45.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.304+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f16180a3bb0 msgr2=0x7f16180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:45.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.304+0000 7f1633e00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f16180a3bb0 0x7f16180a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1624031500 tx=0x7f1624005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:45.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.304+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 msgr2=0x7f163419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:45.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.304+0000 7f1633e00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f163419a840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f162000e9f0 tx=0x7f162000edf0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.305+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 shutdown_connections 2024-04-25T14:59:45.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.305+0000 7f1633e00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f16180a3bb0 0x7f16180a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.305+0000 7f1633e00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1634109d40 0x7f16341a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.305+0000 7f1633e00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1634075f90 0x7f163419a840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.305+0000 7f1633e00640 1 --2- 172.21.15.94:0/2859955803 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16340749b0 0x7f163419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.305+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 >> 172.21.15.94:0/2859955803 conn(0x7f16340fd3f0 msgr2=0x7f1634105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.306+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 shutdown_connections 2024-04-25T14:59:45.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.306+0000 7f1633e00640 1 -- 172.21.15.94:0/2859955803 wait complete. 2024-04-25T14:59:45.343 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T14:59:45.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 -- 172.21.15.94:0/904459443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd4101030 msgr2=0x7f6dd4101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/904459443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd4101030 0x7f6dd4101490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009900 tx=0x7f6dc002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 -- 172.21.15.94:0/904459443 shutdown_connections 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/904459443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41019d0 0x7f6dd4101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/904459443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd4101030 0x7f6dd4101490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/904459443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dd41006d0 0x7f6dd4100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 -- 172.21.15.94:0/904459443 >> 172.21.15.94:0/904459443 conn(0x7f6dd40fc2f0 msgr2=0x7f6dd40fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 -- 172.21.15.94:0/904459443 shutdown_connections 2024-04-25T14:59:45.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.527+0000 7f6dd8a00640 1 -- 172.21.15.94:0/904459443 wait complete. 2024-04-25T14:59:45.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 Processor -- start 2024-04-25T14:59:45.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 -- start start 2024-04-25T14:59:45.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41006d0 0x7f6dd419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:45.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dd4101030 0x7f6dd419a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:45.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd41019d0 0x7f6dd41a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd410fc00 con 0x7f6dd4101030 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd410fa80 con 0x7f6dd41019d0 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.528+0000 7f6dd8a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd410fd80 con 0x7f6dd41006d0 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41006d0 0x7f6dd419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41006d0 0x7f6dd419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:45350/0 (socket says 172.21.15.94:45350) 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 -- 172.21.15.94:0/2436584666 learned_addr learned my addr 172.21.15.94:0/2436584666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd2a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dd4101030 0x7f6dd419a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3e00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd41019d0 0x7f6dd41a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 -- 172.21.15.94:0/2436584666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd41019d0 msgr2=0x7f6dd41a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd41019d0 0x7f6dd41a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 -- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dd4101030 msgr2=0x7f6dd419a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dd4101030 0x7f6dd419a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 -- 172.21.15.94:0/2436584666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd41a1ef0 con 0x7f6dd41006d0 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.529+0000 7f6dd3400640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41006d0 0x7f6dd419a2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400e9f0 tx=0x7f6dc400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:45.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.530+0000 7f6dcbe00640 1 -- 172.21.15.94:0/2436584666 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400cd80 con 0x7f6dd41006d0 2024-04-25T14:59:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.530+0000 7f6dcbe00640 1 -- 172.21.15.94:0/2436584666 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc40108e0 con 0x7f6dd41006d0 2024-04-25T14:59:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.530+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd41a21e0 con 0x7f6dd41006d0 2024-04-25T14:59:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.530+0000 7f6dcbe00640 1 -- 172.21.15.94:0/2436584666 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4018980 con 0x7f6dd41006d0 2024-04-25T14:59:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.530+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd41a2750 con 0x7f6dd41006d0 2024-04-25T14:59:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.531+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd410b420 con 0x7f6dd41006d0 2024-04-25T14:59:45.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.533+0000 7f6dcbe00640 1 -- 172.21.15.94:0/2436584666 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6dc4010430 con 0x7f6dd41006d0 2024-04-25T14:59:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.534+0000 7f6dcbe00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6db80a3c30 0x7f6db80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.534+0000 7f6dcbe00640 1 -- 172.21.15.94:0/2436584666 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f6dc40d05b0 con 0x7f6dd41006d0 2024-04-25T14:59:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.534+0000 7f6dd2a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6db80a3c30 0x7f6db80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:45.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.535+0000 7f6dd2a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6db80a3c30 0x7f6db80a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0031500 tx=0x7f6dc0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:45.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.538+0000 7f6dcbe00640 1 -- 172.21.15.94:0/2436584666 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6dc409ca50 con 0x7f6dd41006d0 2024-04-25T14:59:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:45 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2859955803' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:59:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:45 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2859955803' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:59:45.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:45 smithi094 ceph-mon[27862]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T14:59:45.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:45 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:45.804+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6dd4062b50 con 0x7f6dd41006d0 2024-04-25T14:59:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:45 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2859955803' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:59:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:45 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2859955803' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:59:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:45 smithi138 ceph-mon[32875]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T14:59:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:45 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:45 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2859955803' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T14:59:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:45 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2859955803' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T14:59:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:45 smithi137 ceph-mon[32439]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T14:59:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:45.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:45 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.574+0000 7f6dcbe00640 1 -- 172.21.15.94:0/2436584666 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f6dc40a0d50 con 0x7f6dd41006d0 2024-04-25T14:59:46.575 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T14:59:46.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6db80a3c30 msgr2=0x7f6db80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:46.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6db80a3c30 0x7f6db80a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0031500 tx=0x7f6dc0005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:46.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41006d0 msgr2=0x7f6dd419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:46.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41006d0 0x7f6dd419a2d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400e9f0 tx=0x7f6dc400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 shutdown_connections 2024-04-25T14:59:46.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f6db80a3c30 0x7f6db80a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6dd41019d0 0x7f6dd41a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6dd4101030 0x7f6dd419a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 --2- 172.21.15.94:0/2436584666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6dd41006d0 0x7f6dd419a2d0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400e9f0 tx=0x7f6dc400eeb0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.580+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 >> 172.21.15.94:0/2436584666 conn(0x7f6dd40fc2f0 msgr2=0x7f6dd40fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:46.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.581+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 shutdown_connections 2024-04-25T14:59:46.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.581+0000 7f6dd8a00640 1 -- 172.21.15.94:0/2436584666 wait complete. 2024-04-25T14:59:46.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:59:46.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 -- 172.21.15.94:0/2176584352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c076910 msgr2=0x7f656c076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 --2- 172.21.15.94:0/2176584352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c076910 0x7f656c076d30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6560009900 tx=0x7f656002a400 comp rx=0 tx=0).stop 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 -- 172.21.15.94:0/2176584352 shutdown_connections 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 --2- 172.21.15.94:0/2176584352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c0784f0 0x7f656c078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 --2- 172.21.15.94:0/2176584352 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f656c077b50 0x7f656c077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 --2- 172.21.15.94:0/2176584352 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c076910 0x7f656c076d30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 -- 172.21.15.94:0/2176584352 >> 172.21.15.94:0/2176584352 conn(0x7f656c0ff740 msgr2=0x7f656c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 -- 172.21.15.94:0/2176584352 shutdown_connections 2024-04-25T14:59:46.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.786+0000 7f6574600640 1 -- 172.21.15.94:0/2176584352 wait complete. 2024-04-25T14:59:46.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.787+0000 7f6574600640 1 Processor -- start 2024-04-25T14:59:46.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.787+0000 7f6574600640 1 -- start start 2024-04-25T14:59:46.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6574600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f656c076910 0x7f656c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:46.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6574600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c077b50 0x7f656c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:46.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6574600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c0784f0 0x7f656c1a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:46.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6574600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f656c114130 con 0x7f656c076910 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6574600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f656c113fb0 con 0x7f656c0784f0 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6574600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f656c1142b0 con 0x7f656c077b50 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f656c076910 0x7f656c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c077b50 0x7f656c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c077b50 0x7f656c19ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:45358/0 (socket says 172.21.15.94:45358) 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 -- 172.21.15.94:0/571164240 learned_addr learned my addr 172.21.15.94:0/571164240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6572600640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c0784f0 0x7f656c1a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 -- 172.21.15.94:0/571164240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c0784f0 msgr2=0x7f656c1a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c0784f0 0x7f656c1a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 -- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f656c076910 msgr2=0x7f656c19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f656c076910 0x7f656c19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571200640 1 -- 172.21.15.94:0/571164240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c1a63e0 con 0x7f656c077b50 2024-04-25T14:59:46.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.788+0000 7f6571c00640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f656c076910 0x7f656c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:46.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.789+0000 7f6571200640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c077b50 0x7f656c19ed00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f655800e9f0 tx=0x7f655800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:46.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.789+0000 7f656a000640 1 -- 172.21.15.94:0/571164240 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655800cd80 con 0x7f656c077b50 2024-04-25T14:59:46.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.789+0000 7f656a000640 1 -- 172.21.15.94:0/571164240 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65580108e0 con 0x7f656c077b50 2024-04-25T14:59:46.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.789+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656c1a66d0 con 0x7f656c077b50 2024-04-25T14:59:46.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.789+0000 7f656a000640 1 -- 172.21.15.94:0/571164240 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558018980 con 0x7f656c077b50 2024-04-25T14:59:46.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.789+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656c1a6c20 con 0x7f656c077b50 2024-04-25T14:59:46.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.790+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656c110d00 con 0x7f656c077b50 2024-04-25T14:59:46.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.793+0000 7f656a000640 1 -- 172.21.15.94:0/571164240 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6558010430 con 0x7f656c077b50 2024-04-25T14:59:46.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.795+0000 7f656a000640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65540a3bb0 0x7f65540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:46.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.795+0000 7f656a000640 1 -- 172.21.15.94:0/571164240 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f65580d0620 con 0x7f656c077b50 2024-04-25T14:59:46.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.795+0000 7f6571c00640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65540a3bb0 0x7f65540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:46.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.796+0000 7f6571c00640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65540a3bb0 0x7f65540a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6560031500 tx=0x7f6560005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:46.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.799+0000 7f656a000640 1 -- 172.21.15.94:0/571164240 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f655809ca60 con 0x7f656c077b50 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: pgmap v306: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2436584666' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:59:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:46.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:46.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: pgmap v306: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2436584666' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:46.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:46.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: pgmap v306: 97 pgs: 8 creating+peering, 1 active+clean, 88 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T14:59:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2436584666' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:59:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T14:59:46.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T14:59:46.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.993+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f656c062b50 con 0x7f656c077b50 2024-04-25T14:59:46.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.995+0000 7f656a000640 1 -- 172.21.15.94:0/571164240 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22217 (secure 0 0 0) 0x7f65580a0d60 con 0x7f656c077b50 2024-04-25T14:59:46.996 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:46.996 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":85,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T14:59:45.806139+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:59:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.997+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65540a3bb0 msgr2=0x7f65540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65540a3bb0 0x7f65540a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6560031500 tx=0x7f6560005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c077b50 msgr2=0x7f656c19ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c077b50 0x7f656c19ed00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f655800e9f0 tx=0x7f655800eeb0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 shutdown_connections 2024-04-25T14:59:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65540a3bb0 0x7f65540a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f656c0784f0 0x7f656c1a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f656c077b50 0x7f656c19ed00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 --2- 172.21.15.94:0/571164240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f656c076910 0x7f656c19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 >> 172.21.15.94:0/571164240 conn(0x7f656c0ff740 msgr2=0x7f656c100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 shutdown_connections 2024-04-25T14:59:46.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:46.998+0000 7f6574600640 1 -- 172.21.15.94:0/571164240 wait complete. 2024-04-25T14:59:47.016 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:59:47.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 -- 172.21.15.94:0/3434425110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78102850 msgr2=0x7f0b78104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/3434425110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78102850 0x7f0b78104c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009ca0 tx=0x7f0b6802a8c0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 -- 172.21.15.94:0/3434425110 shutdown_connections 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/3434425110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78107ab0 0x7f0b78109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/3434425110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b78105180 0x7f0b78107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/3434425110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78102850 0x7f0b78104c40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 -- 172.21.15.94:0/3434425110 >> 172.21.15.94:0/3434425110 conn(0x7f0b780fc220 msgr2=0x7f0b780fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 -- 172.21.15.94:0/3434425110 shutdown_connections 2024-04-25T14:59:47.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.206+0000 7f0b7dc00640 1 -- 172.21.15.94:0/3434425110 wait complete. 2024-04-25T14:59:47.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 Processor -- start 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 -- start start 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78102850 0x7f0b7819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b78105180 0x7f0b7819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78107ab0 0x7f0b781a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7810d980 con 0x7f0b78102850 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7810d800 con 0x7f0b78107ab0 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.207+0000 7f0b7dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7810db00 con 0x7f0b78105180 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78102850 0x7f0b7819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78102850 0x7f0b7819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34950/0 (socket says 172.21.15.94:34950) 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 -- 172.21.15.94:0/763180640 learned_addr learned my addr 172.21.15.94:0/763180640 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b7d200640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78107ab0 0x7f0b781a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b6f400640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b78105180 0x7f0b7819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 -- 172.21.15.94:0/763180640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b78105180 msgr2=0x7f0b7819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b78105180 0x7f0b7819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 -- 172.21.15.94:0/763180640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78107ab0 msgr2=0x7f0b781a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78107ab0 0x7f0b781a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 -- 172.21.15.94:0/763180640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b781a1ea0 con 0x7f0b78102850 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b7d200640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78107ab0 0x7f0b781a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b6f400640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b78105180 0x7f0b7819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.208+0000 7f0b77e00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78102850 0x7f0b7819a280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b68005e30 tx=0x7f0b680317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.209+0000 7f0b75600640 1 -- 172.21.15.94:0/763180640 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b68003bf0 con 0x7f0b78102850 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.209+0000 7f0b75600640 1 -- 172.21.15.94:0/763180640 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b68003d90 con 0x7f0b78102850 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.209+0000 7f0b75600640 1 -- 172.21.15.94:0/763180640 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6802fc10 con 0x7f0b78102850 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.209+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b781a2130 con 0x7f0b78102850 2024-04-25T14:59:47.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.209+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b781a2630 con 0x7f0b78102850 2024-04-25T14:59:47.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.210+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b78109600 con 0x7f0b78102850 2024-04-25T14:59:47.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.212+0000 7f0b75600640 1 -- 172.21.15.94:0/763180640 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0b68004040 con 0x7f0b78102850 2024-04-25T14:59:47.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.214+0000 7f0b75600640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0b5c0a3c30 0x7f0b5c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.215+0000 7f0b75600640 1 -- 172.21.15.94:0/763180640 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f0b680f1820 con 0x7f0b78102850 2024-04-25T14:59:47.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.215+0000 7f0b6f400640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0b5c0a3c30 0x7f0b5c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.215+0000 7f0b6f400640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0b5c0a3c30 0x7f0b5c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b7819b790 tx=0x7f0b60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:47.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.217+0000 7f0b75600640 1 -- 172.21.15.94:0/763180640 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b680be340 con 0x7f0b78102850 2024-04-25T14:59:47.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.425+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b78062b50 con 0x7f0b78102850 2024-04-25T14:59:47.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.429+0000 7f0b75600640 1 -- 172.21.15.94:0/763180640 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22204 (secure 0 0 0) 0x7f0b680bdc40 con 0x7f0b78102850 2024-04-25T14:59:47.430 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:47.430 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":86,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T14:59:47.283028+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:59:47.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0b5c0a3c30 msgr2=0x7f0b5c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0b5c0a3c30 0x7f0b5c0a60f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0b7819b790 tx=0x7f0b60005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78102850 msgr2=0x7f0b7819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78102850 0x7f0b7819a280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0b68005e30 tx=0x7f0b680317e0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 shutdown_connections 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0b5c0a3c30 0x7f0b5c0a60f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b78107ab0 0x7f0b781a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b78105180 0x7f0b7819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 --2- 172.21.15.94:0/763180640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b78102850 0x7f0b7819a280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 >> 172.21.15.94:0/763180640 conn(0x7f0b780fc220 msgr2=0x7f0b781020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 shutdown_connections 2024-04-25T14:59:47.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.432+0000 7f0b7dc00640 1 -- 172.21.15.94:0/763180640 wait complete. 2024-04-25T14:59:47.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T14:59:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 -- 172.21.15.94:0/3774754704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105280 msgr2=0x7faf70105700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 --2- 172.21.15.94:0/3774754704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105280 0x7faf70105700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faf60009900 tx=0x7faf6002a410 comp rx=0 tx=0).stop 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 -- 172.21.15.94:0/3774754704 shutdown_connections 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 --2- 172.21.15.94:0/3774754704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf70105c40 0x7faf7010c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 --2- 172.21.15.94:0/3774754704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105280 0x7faf70105700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 --2- 172.21.15.94:0/3774754704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf70104090 0x7faf701044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 -- 172.21.15.94:0/3774754704 >> 172.21.15.94:0/3774754704 conn(0x7faf700ff600 msgr2=0x7faf70101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 -- 172.21.15.94:0/3774754704 shutdown_connections 2024-04-25T14:59:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.665+0000 7faf76a00640 1 -- 172.21.15.94:0/3774754704 wait complete. 2024-04-25T14:59:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 Processor -- start 2024-04-25T14:59:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 -- start start 2024-04-25T14:59:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf70104090 0x7faf7019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf70105280 0x7faf7019a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105c40 0x7faf701a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7faf7010fbd0 con 0x7faf70105c40 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7faf7010fa50 con 0x7faf70105280 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf7010fd50 con 0x7faf70104090 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.666+0000 7faf76000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105c40 0x7faf701a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf76000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105c40 0x7faf701a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34972/0 (socket says 172.21.15.94:34972) 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf75600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf70104090 0x7faf7019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf76000640 1 -- 172.21.15.94:0/2220283406 learned_addr learned my addr 172.21.15.94:0/2220283406 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf74c00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf70105280 0x7faf7019a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf75600640 1 -- 172.21.15.94:0/2220283406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf70105280 msgr2=0x7faf7019a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf75600640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf70105280 0x7faf7019a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf75600640 1 -- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105c40 msgr2=0x7faf701a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf75600640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105c40 0x7faf701a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf75600640 1 -- 172.21.15.94:0/2220283406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf701a1ea0 con 0x7faf70104090 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf76000640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105c40 0x7faf701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf75600640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf70104090 0x7faf7019a280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf701050e0 tx=0x7faf6400dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.667+0000 7faf6e000640 1 -- 172.21.15.94:0/2220283406 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64017070 con 0x7faf70104090 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.668+0000 7faf6e000640 1 -- 172.21.15.94:0/2220283406 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf64009c30 con 0x7faf70104090 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.668+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf701a2190 con 0x7faf70104090 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.668+0000 7faf6e000640 1 -- 172.21.15.94:0/2220283406 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6400f620 con 0x7faf70104090 2024-04-25T14:59:47.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.668+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf701a2700 con 0x7faf70104090 2024-04-25T14:59:47.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.669+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7010b400 con 0x7faf70104090 2024-04-25T14:59:47.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.671+0000 7faf6e000640 1 -- 172.21.15.94:0/2220283406 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf6401a070 con 0x7faf70104090 2024-04-25T14:59:47.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.672+0000 7faf6e000640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7faf580a3c30 0x7faf580a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:47.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.672+0000 7faf6e000640 1 -- 172.21.15.94:0/2220283406 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7faf640cfc90 con 0x7faf70104090 2024-04-25T14:59:47.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.672+0000 7faf74c00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7faf580a3c30 0x7faf580a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:47.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.673+0000 7faf74c00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7faf580a3c30 0x7faf580a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faf60000c00 tx=0x7faf600314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:47.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.677+0000 7faf6e000640 1 -- 172.21.15.94:0/2220283406 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf6409c0d0 con 0x7faf70104090 2024-04-25T14:59:47.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:59:47.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:59:47.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:59:47.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] up:active 2024-04-25T14:59:47.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:47.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/571164240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:47.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T14:59:47.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:47 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/763180640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:47.920+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faf7002d5c0 con 0x7faf70104090 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] up:active 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/571164240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T14:59:47.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:47 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/763180640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:47.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T14:59:47.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T14:59:47.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T14:59:47.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] up:active 2024-04-25T14:59:47.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:47.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/571164240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:47.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T14:59:47.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:47 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/763180640' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:48.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.603+0000 7faf6e000640 1 -- 172.21.15.94:0/2220283406 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7faf640a03d0 con 0x7faf70104090 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7faf580a3c30 msgr2=0x7faf580a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7faf580a3c30 0x7faf580a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faf60000c00 tx=0x7faf600314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf70104090 msgr2=0x7faf7019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf70104090 0x7faf7019a280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf701050e0 tx=0x7faf6400dde0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 shutdown_connections 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7faf580a3c30 0x7faf580a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf70105c40 0x7faf701a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7faf70105280 0x7faf7019a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 --2- 172.21.15.94:0/2220283406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf70104090 0x7faf7019a280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.606+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 >> 172.21.15.94:0/2220283406 conn(0x7faf700ff600 msgr2=0x7faf701085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.607+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 shutdown_connections 2024-04-25T14:59:48.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.607+0000 7faf76a00640 1 -- 172.21.15.94:0/2220283406 wait complete. 2024-04-25T14:59:48.631 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T14:59:48.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 -- 172.21.15.94:0/3505419431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0105210 msgr2=0x7f0cb0107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 --2- 172.21.15.94:0/3505419431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0105210 0x7f0cb0107600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009900 tx=0x7f0c9c02a410 comp rx=0 tx=0).stop 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 -- 172.21.15.94:0/3505419431 shutdown_connections 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 --2- 172.21.15.94:0/3505419431 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb0107b40 0x7f0cb0109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 --2- 172.21.15.94:0/3505419431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0105210 0x7f0cb0107600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 --2- 172.21.15.94:0/3505419431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb01028e0 0x7f0cb0104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 -- 172.21.15.94:0/3505419431 >> 172.21.15.94:0/3505419431 conn(0x7f0cb00fc2f0 msgr2=0x7f0cb00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 -- 172.21.15.94:0/3505419431 shutdown_connections 2024-04-25T14:59:48.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.831+0000 7f0cb6000640 1 -- 172.21.15.94:0/3505419431 wait complete. 2024-04-25T14:59:48.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 Processor -- start 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 -- start start 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb01028e0 0x7f0cb019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb0105210 0x7f0cb019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0107b40 0x7f0cb01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010d9d0 con 0x7f0cb0107b40 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010d850 con 0x7f0cb01028e0 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.832+0000 7f0cb6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb010db50 con 0x7f0cb0105210 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0107b40 0x7f0cb01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:48.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0107b40 0x7f0cb01a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34998/0 (socket says 172.21.15.94:34998) 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 -- 172.21.15.94:0/2744111524 learned_addr learned my addr 172.21.15.94:0/2744111524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb4c00640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb01028e0 0x7f0cb019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cafe00640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb0105210 0x7f0cb019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 -- 172.21.15.94:0/2744111524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb0105210 msgr2=0x7f0cb019a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb0105210 0x7f0cb019a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 -- 172.21.15.94:0/2744111524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb01028e0 msgr2=0x7f0cb019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb01028e0 0x7f0cb019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 -- 172.21.15.94:0/2744111524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb01a1ef0 con 0x7f0cb0107b40 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb4c00640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb01028e0 0x7f0cb019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:48.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.833+0000 7f0cb5600640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0107b40 0x7f0cb01a1880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800eb10 tx=0x7f0ca800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:48.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.834+0000 7f0cad600640 1 -- 172.21.15.94:0/2744111524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8004b20 con 0x7f0cb0107b40 2024-04-25T14:59:48.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.834+0000 7f0cad600640 1 -- 172.21.15.94:0/2744111524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ca8004cc0 con 0x7f0cb0107b40 2024-04-25T14:59:48.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.834+0000 7f0cad600640 1 -- 172.21.15.94:0/2744111524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8010670 con 0x7f0cb0107b40 2024-04-25T14:59:48.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.834+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb01a21e0 con 0x7f0cb0107b40 2024-04-25T14:59:48.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.834+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb01a26b0 con 0x7f0cb0107b40 2024-04-25T14:59:48.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.835+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb0109690 con 0x7f0cb0107b40 2024-04-25T14:59:48.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.837+0000 7f0cad600640 1 -- 172.21.15.94:0/2744111524 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ca8005020 con 0x7f0cb0107b40 2024-04-25T14:59:48.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.839+0000 7f0cad600640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0c980a3bb0 0x7f0c980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:48.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.840+0000 7f0cad600640 1 -- 172.21.15.94:0/2744111524 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0ca80d09b0 con 0x7f0cb0107b40 2024-04-25T14:59:48.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.840+0000 7f0cb4c00640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0c980a3bb0 0x7f0c980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:48.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.840+0000 7f0cb4c00640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0c980a3bb0 0x7f0c980a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40095f0 tx=0x7f0ca4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:48.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:48.844+0000 7f0cad600640 1 -- 172.21.15.94:0/2744111524 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ca809d440 con 0x7f0cb0107b40 2024-04-25T14:59:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:48 smithi138 ceph-mon[32875]: pgmap v309: 97 pgs: 21 creating+peering, 37 active+clean, 39 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T14:59:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:48 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2220283406' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:59:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:48 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:59:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:48 smithi138 ceph-mon[32875]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:59:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:48 smithi138 ceph-mon[32875]: Cluster is now healthy 2024-04-25T14:59:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:48 smithi094 ceph-mon[27862]: pgmap v309: 97 pgs: 21 creating+peering, 37 active+clean, 39 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T14:59:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:48 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2220283406' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:59:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:48 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:59:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:48 smithi094 ceph-mon[27862]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:59:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:48 smithi094 ceph-mon[27862]: Cluster is now healthy 2024-04-25T14:59:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:48 smithi137 ceph-mon[32439]: pgmap v309: 97 pgs: 21 creating+peering, 37 active+clean, 39 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T14:59:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:48 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2220283406' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:59:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:48 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T14:59:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:48 smithi137 ceph-mon[32439]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T14:59:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:48 smithi137 ceph-mon[32439]: Cluster is now healthy 2024-04-25T14:59:49.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.087+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0cb0062b50 con 0x7f0cb0107b40 2024-04-25T14:59:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.615+0000 7f0cad600640 1 -- 172.21.15.94:0/2744111524 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f0ca809cd40 con 0x7f0cb0107b40 2024-04-25T14:59:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.618+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0c980a3bb0 msgr2=0x7f0c980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.618+0000 7f0cb6000640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0c980a3bb0 0x7f0c980a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40095f0 tx=0x7f0ca4005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.618+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0107b40 msgr2=0x7f0cb01a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.618+0000 7f0cb6000640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0107b40 0x7f0cb01a1880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ca800eb10 tx=0x7f0ca800c2d0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 shutdown_connections 2024-04-25T14:59:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0c980a3bb0 0x7f0c980a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0cb0107b40 0x7f0cb01a1880 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cb0105210 0x7f0cb019a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 --2- 172.21.15.94:0/2744111524 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0cb01028e0 0x7f0cb019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 >> 172.21.15.94:0/2744111524 conn(0x7f0cb00fc2f0 msgr2=0x7f0cb0102080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:49.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 shutdown_connections 2024-04-25T14:59:49.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.619+0000 7f0cb6000640 1 -- 172.21.15.94:0/2744111524 wait complete. 2024-04-25T14:59:49.644 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T14:59:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 -- 172.21.15.94:0/954170751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc101e60 msgr2=0x7f3ffc1022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 --2- 172.21.15.94:0/954170751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc101e60 0x7f3ffc1022e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009900 tx=0x7f3fe802a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 -- 172.21.15.94:0/954170751 shutdown_connections 2024-04-25T14:59:49.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 --2- 172.21.15.94:0/954170751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ffc102820 0x7f3ffc1090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 --2- 172.21.15.94:0/954170751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc101e60 0x7f3ffc1022e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 --2- 172.21.15.94:0/954170751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ffc100c70 0x7f3ffc101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 -- 172.21.15.94:0/954170751 >> 172.21.15.94:0/954170751 conn(0x7f3ffc0fc240 msgr2=0x7f3ffc0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:49.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 -- 172.21.15.94:0/954170751 shutdown_connections 2024-04-25T14:59:49.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.834+0000 7f4002000640 1 -- 172.21.15.94:0/954170751 wait complete. 2024-04-25T14:59:49.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 Processor -- start 2024-04-25T14:59:49.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 -- start start 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc100c70 0x7f3ffc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ffc101e60 0x7f3ffc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ffc102820 0x7f3ffc19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc10c7d0 con 0x7f3ffc100c70 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc10c650 con 0x7f3ffc102820 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.835+0000 7f4002000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc10c950 con 0x7f3ffc101e60 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc100c70 0x7f3ffc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc100c70 0x7f3ffc197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35018/0 (socket says 172.21.15.94:35018) 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 -- 172.21.15.94:0/2539138716 learned_addr learned my addr 172.21.15.94:0/2539138716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f3ffbe00640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ffc101e60 0x7f3ffc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:49.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4001600640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ffc102820 0x7f3ffc19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 -- 172.21.15.94:0/2539138716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ffc101e60 msgr2=0x7f3ffc1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ffc101e60 0x7f3ffc1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 -- 172.21.15.94:0/2539138716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ffc102820 msgr2=0x7f3ffc19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ffc102820 0x7f3ffc19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.836+0000 7f4000c00640 1 -- 172.21.15.94:0/2539138716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc19fbb0 con 0x7f3ffc100c70 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.837+0000 7f4000c00640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc100c70 0x7f3ffc197f90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000d950 tx=0x7f3ff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.837+0000 7f3ff9600640 1 -- 172.21.15.94:0/2539138716 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0017020 con 0x7f3ffc100c70 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.837+0000 7f3ff9600640 1 -- 172.21.15.94:0/2539138716 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff0009c30 con 0x7f3ffc100c70 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.837+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc19fea0 con 0x7f3ffc100c70 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.837+0000 7f3ff9600640 1 -- 172.21.15.94:0/2539138716 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000f6b0 con 0x7f3ffc100c70 2024-04-25T14:59:49.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.837+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a0310 con 0x7f3ffc100c70 2024-04-25T14:59:49.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.838+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc1a0600 con 0x7f3ffc100c70 2024-04-25T14:59:49.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.840+0000 7f3ff9600640 1 -- 172.21.15.94:0/2539138716 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ff001a020 con 0x7f3ffc100c70 2024-04-25T14:59:49.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.842+0000 7f3ff9600640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3fe40a3c30 0x7f3fe40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:49.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.842+0000 7f3ff9600640 1 -- 172.21.15.94:0/2539138716 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3ff00d0670 con 0x7f3ffc100c70 2024-04-25T14:59:49.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.842+0000 7f3ffbe00640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3fe40a3c30 0x7f3fe40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:49.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.843+0000 7f3ffbe00640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3fe40a3c30 0x7f3fe40a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8031500 tx=0x7f3fe8005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:49.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:49.844+0000 7f3ff9600640 1 -- 172.21.15.94:0/2539138716 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ff0009de0 con 0x7f3ffc100c70 2024-04-25T14:59:49.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:49 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:59:49.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:49 smithi138 ceph-mon[32875]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:49.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:49 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2744111524' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:59:49.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:49 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:59:49.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:49 smithi094 ceph-mon[27862]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:49.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:49 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2744111524' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:59:49.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:49 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T14:59:49.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:49 smithi137 ceph-mon[32439]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:49.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:49 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2744111524' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T14:59:50.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.087+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f3ffc1a09f0 con 0x7f3ffc100c70 2024-04-25T14:59:50.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.628+0000 7f3ff9600640 1 -- 172.21.15.94:0/2539138716 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f3ff009d180 con 0x7f3ffc100c70 2024-04-25T14:59:50.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.632+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3fe40a3c30 msgr2=0x7f3fe40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:50.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.632+0000 7f4002000640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3fe40a3c30 0x7f3fe40a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8031500 tx=0x7f3fe8005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:50.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc100c70 msgr2=0x7f3ffc197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc100c70 0x7f3ffc197f90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000d950 tx=0x7f3ff000de10 comp rx=0 tx=0).stop 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 shutdown_connections 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3fe40a3c30 0x7f3fe40a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3ffc102820 0x7f3ffc19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ffc101e60 0x7f3ffc1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 --2- 172.21.15.94:0/2539138716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ffc100c70 0x7f3ffc197f90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 >> 172.21.15.94:0/2539138716 conn(0x7f3ffc0fc240 msgr2=0x7f3ffc0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:50.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.633+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 shutdown_connections 2024-04-25T14:59:50.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.634+0000 7f4002000640 1 -- 172.21.15.94:0/2539138716 wait complete. 2024-04-25T14:59:50.656 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T14:59:50.656 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T14:59:50.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.859+0000 7f41e0a00640 1 -- 172.21.15.94:0/925327492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 msgr2=0x7f41dc102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:50.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.859+0000 7f41e0a00640 1 --2- 172.21.15.94:0/925327492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc102280 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009900 tx=0x7f41cc02a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.859+0000 7f41e0a00640 1 -- 172.21.15.94:0/925327492 shutdown_connections 2024-04-25T14:59:50.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.860+0000 7f41e0a00640 1 --2- 172.21.15.94:0/925327492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41dc103970 0x7f41dc10a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.860+0000 7f41e0a00640 1 --2- 172.21.15.94:0/925327492 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41dc102fb0 0x7f41dc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.860+0000 7f41e0a00640 1 --2- 172.21.15.94:0/925327492 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc102280 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.860+0000 7f41e0a00640 1 -- 172.21.15.94:0/925327492 >> 172.21.15.94:0/925327492 conn(0x7f41dc0fd3f0 msgr2=0x7f41dc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:50.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.860+0000 7f41e0a00640 1 -- 172.21.15.94:0/925327492 shutdown_connections 2024-04-25T14:59:50.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.860+0000 7f41e0a00640 1 -- 172.21.15.94:0/925327492 wait complete. 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 Processor -- start 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 -- start start 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41dc102fb0 0x7f41dc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41dc103970 0x7f41dc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc10d990 con 0x7f41dc101e60 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc10d810 con 0x7f41dc102fb0 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.861+0000 7f41e0a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc10db10 con 0x7f41dc103970 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35042/0 (socket says 172.21.15.94:35042) 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 -- 172.21.15.94:0/895336384 learned_addr learned my addr 172.21.15.94:0/895336384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:50.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41dbe00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41dc103970 0x7f41dc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:50.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41daa00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41dc102fb0 0x7f41dc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:50.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 -- 172.21.15.94:0/895336384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41dc103970 msgr2=0x7f41dc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:50.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41dc103970 0x7f41dc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 -- 172.21.15.94:0/895336384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41dc102fb0 msgr2=0x7f41dc1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:50.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41dc102fb0 0x7f41dc1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:50.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 -- 172.21.15.94:0/895336384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc19fc90 con 0x7f41dc101e60 2024-04-25T14:59:50.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.862+0000 7f41db400640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc198070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41cc005940 tx=0x7f41cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:50.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.863+0000 7f41d3e00640 1 -- 172.21.15.94:0/895336384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc03f030 con 0x7f41dc101e60 2024-04-25T14:59:50.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.863+0000 7f41d3e00640 1 -- 172.21.15.94:0/895336384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41cc003cb0 con 0x7f41dc101e60 2024-04-25T14:59:50.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.863+0000 7f41d3e00640 1 -- 172.21.15.94:0/895336384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc035070 con 0x7f41dc101e60 2024-04-25T14:59:50.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.863+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc19ff20 con 0x7f41dc101e60 2024-04-25T14:59:50.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.863+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc1a0380 con 0x7f41dc101e60 2024-04-25T14:59:50.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.864+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc1091f0 con 0x7f41dc101e60 2024-04-25T14:59:50.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.866+0000 7f41d3e00640 1 -- 172.21.15.94:0/895336384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f41cc042030 con 0x7f41dc101e60 2024-04-25T14:59:50.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.868+0000 7f41d3e00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f41c00a3c30 0x7f41c00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:50.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.868+0000 7f41d3e00640 1 -- 172.21.15.94:0/895336384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f41cc0f0cb0 con 0x7f41dc101e60 2024-04-25T14:59:50.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.868+0000 7f41daa00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f41c00a3c30 0x7f41c00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:50.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.869+0000 7f41daa00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f41c00a3c30 0x7f41c00a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41dc199580 tx=0x7f41c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:50.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:50.870+0000 7f41d3e00640 1 -- 172.21.15.94:0/895336384 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41cc003e60 con 0x7f41dc101e60 2024-04-25T14:59:50.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:50 smithi094 ceph-mon[27862]: pgmap v310: 97 pgs: 18 creating+peering, 65 active+clean, 14 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-25T14:59:50.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:50 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2744111524' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:59:50.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:50 smithi094 ceph-mon[27862]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:50.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:50 smithi094 ceph-mon[27862]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T14:59:50.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:50 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2539138716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:59:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:50 smithi138 ceph-mon[32875]: pgmap v310: 97 pgs: 18 creating+peering, 65 active+clean, 14 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-25T14:59:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:50 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2744111524' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:59:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:50 smithi138 ceph-mon[32875]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:50 smithi138 ceph-mon[32875]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T14:59:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:50 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2539138716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:59:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:50 smithi137 ceph-mon[32439]: pgmap v310: 97 pgs: 18 creating+peering, 65 active+clean, 14 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-04-25T14:59:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:50 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2744111524' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T14:59:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:50 smithi137 ceph-mon[32439]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T14:59:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:50 smithi137 ceph-mon[32439]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T14:59:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:50 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2539138716' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T14:59:51.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.104+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f41dc062390 con 0x7f41c00a3c30 2024-04-25T14:59:51.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.136+0000 7f41d3e00640 1 -- 172.21.15.94:0/895336384 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41dc062390 con 0x7f41c00a3c30 2024-04-25T14:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f41c00a3c30 msgr2=0x7f41c00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f41c00a3c30 0x7f41c00a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41dc199580 tx=0x7f41c8005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 msgr2=0x7f41dc198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc198070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f41cc005940 tx=0x7f41cc0042f0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 shutdown_connections 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f41c00a3c30 0x7f41c00a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41dc103970 0x7f41dc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f41dc102fb0 0x7f41dc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 --2- 172.21.15.94:0/895336384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41dc101e60 0x7f41dc198070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 >> 172.21.15.94:0/895336384 conn(0x7f41dc0fd3f0 msgr2=0x7f41dc1063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 shutdown_connections 2024-04-25T14:59:51.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.139+0000 7f41e0a00640 1 -- 172.21.15.94:0/895336384 wait complete. 2024-04-25T14:59:51.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T14:59:51.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:59:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:59:51.117+0000 7fef22400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:59:51.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:59:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:59:51.117+0000 7fef22400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:59:51.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:59:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:59:51.117+0000 7fef22400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:59:51.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:59:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:59:51.117+0000 7fef22400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:59:51.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 14:59:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T14:59:51.117+0000 7fef22400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 -- 172.21.15.94:0/1663100959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88109cf0 msgr2=0x7f3b8810c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1663100959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88109cf0 0x7f3b8810c190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009be0 tx=0x7f3b8002a750 comp rx=0 tx=0).stop 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 -- 172.21.15.94:0/1663100959 shutdown_connections 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1663100959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88109cf0 0x7f3b8810c190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1663100959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88074950 0x7f3b88074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1663100959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88075f30 0x7f3b88074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 -- 172.21.15.94:0/1663100959 >> 172.21.15.94:0/1663100959 conn(0x7f3b880fd3f0 msgr2=0x7f3b880ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:51.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 -- 172.21.15.94:0/1663100959 shutdown_connections 2024-04-25T14:59:51.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.333+0000 7f3b8d400640 1 -- 172.21.15.94:0/1663100959 wait complete. 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 Processor -- start 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 -- start start 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88074950 0x7f3b8819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88075f30 0x7f3b8819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88109cf0 0x7f3b881a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810fba0 con 0x7f3b88074950 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810fa20 con 0x7f3b88109cf0 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.334+0000 7f3b8d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b8810fd20 con 0x7f3b88075f30 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88074950 0x7f3b8819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88074950 0x7f3b8819a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35090/0 (socket says 172.21.15.94:35090) 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88075f30 0x7f3b8819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 -- 172.21.15.94:0/1361470094 learned_addr learned my addr 172.21.15.94:0/1361470094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b8ca00640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88109cf0 0x7f3b881a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 -- 172.21.15.94:0/1361470094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88075f30 msgr2=0x7f3b8819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88075f30 0x7f3b8819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 -- 172.21.15.94:0/1361470094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88109cf0 msgr2=0x7f3b881a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88109cf0 0x7f3b881a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 -- 172.21.15.94:0/1361470094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b881a1e60 con 0x7f3b88074950 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.335+0000 7f3b87e00640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88074950 0x7f3b8819a240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800e9c0 tx=0x7f3b7800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.336+0000 7f3b84c00640 1 -- 172.21.15.94:0/1361470094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800cde0 con 0x7f3b88074950 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.336+0000 7f3b84c00640 1 -- 172.21.15.94:0/1361470094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b780035f0 con 0x7f3b88074950 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.336+0000 7f3b84c00640 1 -- 172.21.15.94:0/1361470094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78010690 con 0x7f3b88074950 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.336+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b881a2150 con 0x7f3b88074950 2024-04-25T14:59:51.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.336+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b881a26a0 con 0x7f3b88074950 2024-04-25T14:59:51.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.337+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b88108ab0 con 0x7f3b88074950 2024-04-25T14:59:51.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.339+0000 7f3b84c00640 1 -- 172.21.15.94:0/1361470094 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3b78004000 con 0x7f3b88074950 2024-04-25T14:59:51.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.340+0000 7f3b84c00640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3b6c0a3bb0 0x7f3b6c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.340+0000 7f3b84c00640 1 -- 172.21.15.94:0/1361470094 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3b780d0640 con 0x7f3b88074950 2024-04-25T14:59:51.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.340+0000 7f3b87400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3b6c0a3bb0 0x7f3b6c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.341+0000 7f3b87400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3b6c0a3bb0 0x7f3b6c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b8819b750 tx=0x7f3b74005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:51.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.344+0000 7f3b84c00640 1 -- 172.21.15.94:0/1361470094 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b78009510 con 0x7f3b88074950 2024-04-25T14:59:51.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.577+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3b88062390 con 0x7f3b6c0a3bb0 2024-04-25T14:59:51.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.614+0000 7f3b84c00640 1 -- 172.21.15.94:0/1361470094 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3b88062390 con 0x7f3b6c0a3bb0 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3b6c0a3bb0 msgr2=0x7f3b6c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3b6c0a3bb0 0x7f3b6c0a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b8819b750 tx=0x7f3b74005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88074950 msgr2=0x7f3b8819a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88074950 0x7f3b8819a240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800e9c0 tx=0x7f3b7800ee80 comp rx=0 tx=0).stop 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 shutdown_connections 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3b6c0a3bb0 0x7f3b6c0a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3b88109cf0 0x7f3b881a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b88075f30 0x7f3b8819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 --2- 172.21.15.94:0/1361470094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3b88074950 0x7f3b8819a240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.616+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 >> 172.21.15.94:0/1361470094 conn(0x7f3b880fd3f0 msgr2=0x7f3b88105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.617+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 shutdown_connections 2024-04-25T14:59:51.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.617+0000 7f3b8d400640 1 -- 172.21.15.94:0/1361470094 wait complete. 2024-04-25T14:59:51.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T14:59:51.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 -- 172.21.15.94:0/3431949369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 msgr2=0x7f9c78103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/3431949369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c78103430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009900 tx=0x7f9c6402a410 comp rx=0 tx=0).stop 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 -- 172.21.15.94:0/3431949369 shutdown_connections 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/3431949369 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78103970 0x7f9c7810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/3431949369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c78103430 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/3431949369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c78101e60 0x7f9c78102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 -- 172.21.15.94:0/3431949369 >> 172.21.15.94:0/3431949369 conn(0x7f9c780fd3f0 msgr2=0x7f9c780ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 -- 172.21.15.94:0/3431949369 shutdown_connections 2024-04-25T14:59:51.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.821+0000 7f9c7dc00640 1 -- 172.21.15.94:0/3431949369 wait complete. 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 Processor -- start 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 -- start start 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78101e60 0x7f9c7819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c7819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c78103970 0x7f9c781a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810d9b0 con 0x7f9c78102fb0 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810d830 con 0x7f9c78101e60 2024-04-25T14:59:51.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.822+0000 7f9c7dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7810db30 con 0x7f9c78103970 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c7819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c7819a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35106/0 (socket says 172.21.15.94:35106) 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 -- 172.21.15.94:0/877520741 learned_addr learned my addr 172.21.15.94:0/877520741 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c7d200640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c78103970 0x7f9c781a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77e00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78101e60 0x7f9c7819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 -- 172.21.15.94:0/877520741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c78103970 msgr2=0x7f9c781a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c78103970 0x7f9c781a18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 -- 172.21.15.94:0/877520741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78101e60 msgr2=0x7f9c7819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78101e60 0x7f9c7819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 -- 172.21.15.94:0/877520741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c781a1f10 con 0x7f9c78102fb0 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77e00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78101e60 0x7f9c7819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.823+0000 7f9c77400640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c7819a830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006010 tx=0x7f9c640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:51.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.824+0000 7f9c74c00640 1 -- 172.21.15.94:0/877520741 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6403f030 con 0x7f9c78102fb0 2024-04-25T14:59:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.824+0000 7f9c74c00640 1 -- 172.21.15.94:0/877520741 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c64031910 con 0x7f9c78102fb0 2024-04-25T14:59:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.824+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c781a21a0 con 0x7f9c78102fb0 2024-04-25T14:59:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.824+0000 7f9c74c00640 1 -- 172.21.15.94:0/877520741 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64035070 con 0x7f9c78102fb0 2024-04-25T14:59:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.824+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c781a2690 con 0x7f9c78102fb0 2024-04-25T14:59:51.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.825+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c781a29b0 con 0x7f9c78102fb0 2024-04-25T14:59:51.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.827+0000 7f9c74c00640 1 -- 172.21.15.94:0/877520741 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9c64042030 con 0x7f9c78102fb0 2024-04-25T14:59:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.828+0000 7f9c74c00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9c5c0a3c40 0x7f9c5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.828+0000 7f9c74c00640 1 -- 172.21.15.94:0/877520741 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9c640f0980 con 0x7f9c78102fb0 2024-04-25T14:59:51.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.828+0000 7f9c77e00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9c5c0a3c40 0x7f9c5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:51.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.829+0000 7f9c77e00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9c5c0a3c40 0x7f9c5c0a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009c20 tx=0x7f9c68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:51.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:51.832+0000 7f9c74c00640 1 -- 172.21.15.94:0/877520741 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c640bd2b0 con 0x7f9c78102fb0 2024-04-25T14:59:51.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:51 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2539138716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:59:51.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:51 smithi138 ceph-mon[32875]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T14:59:51.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:51 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2539138716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:59:51.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:51 smithi094 ceph-mon[27862]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T14:59:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:51 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2539138716' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T14:59:51.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:51 smithi137 ceph-mon[32439]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-04-25T14:59:51.983 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 14:59:51 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b[63292]: 2024-04-25T14:59:51.563+0000 7f54e28ab700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T14:59:52.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.048+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9c780758a0 con 0x7f9c5c0a3c40 2024-04-25T14:59:52.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.057+0000 7f9c74c00640 1 -- 172.21.15.94:0/877520741 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9c780758a0 con 0x7f9c5c0a3c40 2024-04-25T14:59:52.057 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_0/26b4f823-fc93-4daf-bd3d-4bc85d578e7b 2024-04-25T14:59:52.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9c5c0a3c40 msgr2=0x7f9c5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9c5c0a3c40 0x7f9c5c0a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009c20 tx=0x7f9c68005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 msgr2=0x7f9c7819a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c7819a830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c64006010 tx=0x7f9c640042f0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 shutdown_connections 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9c5c0a3c40 0x7f9c5c0a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c78103970 0x7f9c781a18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78102fb0 0x7f9c7819a830 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 --2- 172.21.15.94:0/877520741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9c78101e60 0x7f9c7819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 >> 172.21.15.94:0/877520741 conn(0x7f9c780fd3f0 msgr2=0x7f9c78074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 shutdown_connections 2024-04-25T14:59:52.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.059+0000 7f9c7dc00640 1 -- 172.21.15.94:0/877520741 wait complete. 2024-04-25T14:59:52.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T14:59:52.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 -- 172.21.15.94:0/721309740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac108490 msgr2=0x7fe1ac1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 --2- 172.21.15.94:0/721309740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac108490 0x7fe1ac1088b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009900 tx=0x7fe1a002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 -- 172.21.15.94:0/721309740 shutdown_connections 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 --2- 172.21.15.94:0/721309740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac10a040 0x7fe1ac1108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 --2- 172.21.15.94:0/721309740 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac109680 0x7fe1ac109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 --2- 172.21.15.94:0/721309740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac108490 0x7fe1ac1088b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 -- 172.21.15.94:0/721309740 >> 172.21.15.94:0/721309740 conn(0x7fe1ac077420 msgr2=0x7fe1ac079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 -- 172.21.15.94:0/721309740 shutdown_connections 2024-04-25T14:59:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.285+0000 7fe1b3200640 1 -- 172.21.15.94:0/721309740 wait complete. 2024-04-25T14:59:52.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 Processor -- start 2024-04-25T14:59:52.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 -- start start 2024-04-25T14:59:52.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac108490 0x7fe1ac19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac109680 0x7fe1ac19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac10a040 0x7fe1ac1a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac113fd0 con 0x7fe1ac10a040 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac113e50 con 0x7fe1ac109680 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.286+0000 7fe1b3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac114150 con 0x7fe1ac108490 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac10a040 0x7fe1ac1a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac108490 0x7fe1ac19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac10a040 0x7fe1ac1a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35118/0 (socket says 172.21.15.94:35118) 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac108490 0x7fe1ac19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42608/0 (socket says 172.21.15.94:42608) 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b2800640 1 -- 172.21.15.94:0/393971141 learned_addr learned my addr 172.21.15.94:0/393971141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1400640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac109680 0x7fe1ac19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1e00640 1 -- 172.21.15.94:0/393971141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac109680 msgr2=0x7fe1ac19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1e00640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac109680 0x7fe1ac19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1e00640 1 -- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac10a040 msgr2=0x7fe1ac1a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1e00640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac10a040 0x7fe1ac1a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b1e00640 1 -- 172.21.15.94:0/393971141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac1a6280 con 0x7fe1ac108490 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.287+0000 7fe1b2800640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac10a040 0x7fe1ac1a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.288+0000 7fe1b1e00640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac108490 0x7fe1ac19e660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006010 tx=0x7fe1a00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.288+0000 7fe1a6a00640 1 -- 172.21.15.94:0/393971141 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a003f030 con 0x7fe1ac108490 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.288+0000 7fe1a6a00640 1 -- 172.21.15.94:0/393971141 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1a0031910 con 0x7fe1ac108490 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.288+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac1a6510 con 0x7fe1ac108490 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.288+0000 7fe1a6a00640 1 -- 172.21.15.94:0/393971141 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0035070 con 0x7fe1ac108490 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.288+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac1a6a00 con 0x7fe1ac108490 2024-04-25T14:59:52.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.289+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ac10f7d0 con 0x7fe1ac108490 2024-04-25T14:59:52.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.291+0000 7fe1a6a00640 1 -- 172.21.15.94:0/393971141 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe1a0042030 con 0x7fe1ac108490 2024-04-25T14:59:52.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.292+0000 7fe1a6a00640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe1940a3c40 0x7fe1940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.292+0000 7fe1a6a00640 1 -- 172.21.15.94:0/393971141 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe1a00f0c40 con 0x7fe1ac108490 2024-04-25T14:59:52.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.293+0000 7fe1b1400640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe1940a3c40 0x7fe1940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.293+0000 7fe1b1400640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe1940a3c40 0x7fe1940a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0095f0 tx=0x7fe19c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:52.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.295+0000 7fe1a6a00640 1 -- 172.21.15.94:0/393971141 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe1a00bcf20 con 0x7fe1ac108490 2024-04-25T14:59:52.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.526+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe1ac062420 con 0x7fe1940a3c40 2024-04-25T14:59:52.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.561+0000 7fe1a6a00640 1 -- 172.21.15.94:0/393971141 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe1ac062420 con 0x7fe1940a3c40 2024-04-25T14:59:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe1940a3c40 msgr2=0x7fe1940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe1940a3c40 0x7fe1940a6100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0095f0 tx=0x7fe19c005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac108490 msgr2=0x7fe1ac19e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac108490 0x7fe1ac19e660 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0006010 tx=0x7fe1a00042f0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 shutdown_connections 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe1940a3c40 0x7fe1940a6100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe1ac10a040 0x7fe1ac1a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe1ac109680 0x7fe1ac19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 --2- 172.21.15.94:0/393971141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe1ac108490 0x7fe1ac19e660 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 >> 172.21.15.94:0/393971141 conn(0x7fe1ac077420 msgr2=0x7fe1ac10c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 shutdown_connections 2024-04-25T14:59:52.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.564+0000 7fe1b3200640 1 -- 172.21.15.94:0/393971141 wait complete. 2024-04-25T14:59:52.587 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T14:59:52.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 -- 172.21.15.94:0/2963045581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0105220 msgr2=0x7ff6c0107610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/2963045581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0105220 0x7ff6c0107610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009900 tx=0x7ff6b002a400 comp rx=0 tx=0).stop 2024-04-25T14:59:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 -- 172.21.15.94:0/2963045581 shutdown_connections 2024-04-25T14:59:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/2963045581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0107b50 0x7ff6c0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/2963045581 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0105220 0x7ff6c0107610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/2963045581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 0x7ff6c0104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 -- 172.21.15.94:0/2963045581 >> 172.21.15.94:0/2963045581 conn(0x7ff6c00fc300 msgr2=0x7ff6c00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 -- 172.21.15.94:0/2963045581 shutdown_connections 2024-04-25T14:59:52.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.785+0000 7ff6c4c00640 1 -- 172.21.15.94:0/2963045581 wait complete. 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.786+0000 7ff6c4c00640 1 Processor -- start 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.786+0000 7ff6c4c00640 1 -- start start 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6c4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 0x7ff6c019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6c4c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0105220 0x7ff6c019a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6c4c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0107b50 0x7ff6c01a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6c4c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c010d990 con 0x7ff6c0105220 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6c4c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c010d810 con 0x7ff6c0107b50 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6c4c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c010db10 con 0x7ff6c01028f0 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 0x7ff6c019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 0x7ff6c019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42624/0 (socket says 172.21.15.94:42624) 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bf400640 1 -- 172.21.15.94:0/3384422614 learned_addr learned my addr 172.21.15.94:0/3384422614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bea00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0105220 0x7ff6c019a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bfe00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0107b50 0x7ff6c01a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bf400640 1 -- 172.21.15.94:0/3384422614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0107b50 msgr2=0x7ff6c01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bf400640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0107b50 0x7ff6c01a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bf400640 1 -- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0105220 msgr2=0x7ff6c019a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:52.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.787+0000 7ff6bf400640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0105220 0x7ff6c019a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:52.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.788+0000 7ff6bf400640 1 -- 172.21.15.94:0/3384422614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c01a1f40 con 0x7ff6c01028f0 2024-04-25T14:59:52.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.788+0000 7ff6bf400640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 0x7ff6c019a320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00ea60 tx=0x7ff6ac00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.788+0000 7ff6b7e00640 1 -- 172.21.15.94:0/3384422614 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac00cd20 con 0x7ff6c01028f0 2024-04-25T14:59:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.788+0000 7ff6b7e00640 1 -- 172.21.15.94:0/3384422614 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac0035f0 con 0x7ff6c01028f0 2024-04-25T14:59:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.788+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c01a2230 con 0x7ff6c01028f0 2024-04-25T14:59:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.788+0000 7ff6b7e00640 1 -- 172.21.15.94:0/3384422614 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac010690 con 0x7ff6c01028f0 2024-04-25T14:59:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.788+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c01a27a0 con 0x7ff6c01028f0 2024-04-25T14:59:52.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.789+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6c01096a0 con 0x7ff6c01028f0 2024-04-25T14:59:52.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.792+0000 7ff6b7e00640 1 -- 172.21.15.94:0/3384422614 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff6ac0037a0 con 0x7ff6c01028f0 2024-04-25T14:59:52.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.793+0000 7ff6b7e00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6a40a3cc0 0x7ff6a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.794+0000 7ff6b7e00640 1 -- 172.21.15.94:0/3384422614 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff6ac0d06b0 con 0x7ff6c01028f0 2024-04-25T14:59:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.794+0000 7ff6bea00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6a40a3cc0 0x7ff6a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:52.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.794+0000 7ff6bea00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6a40a3cc0 0x7ff6a40a6180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009230 tx=0x7ff6b00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:52.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:52.798+0000 7ff6b7e00640 1 -- 172.21.15.94:0/3384422614 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6ac09cfe0 con 0x7ff6c01028f0 2024-04-25T14:59:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:52 smithi138 ceph-mon[32875]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:52 smithi138 ceph-mon[32875]: pgmap v311: 97 pgs: 97 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T14:59:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:52 smithi138 ceph-mon[32875]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:52 smithi138 ceph-mon[32875]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T14:59:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:52 smithi094 ceph-mon[27862]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:52 smithi094 ceph-mon[27862]: pgmap v311: 97 pgs: 97 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T14:59:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:52 smithi094 ceph-mon[27862]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:52 smithi094 ceph-mon[27862]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T14:59:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:52 smithi137 ceph-mon[32439]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:52 smithi137 ceph-mon[32439]: pgmap v311: 97 pgs: 97 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 331 B/s rd, 1.8 KiB/s wr, 6 op/s 2024-04-25T14:59:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:52 smithi137 ceph-mon[32439]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:52 smithi137 ceph-mon[32439]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T14:59:53.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.033+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff6c0062390 con 0x7ff6a40a3cc0 2024-04-25T14:59:53.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.042+0000 7ff6b7e00640 1 -- 172.21.15.94:0/3384422614 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff6c0062390 con 0x7ff6a40a3cc0 2024-04-25T14:59:53.043 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_1/8662f957-a607-4fca-9c75-81aedaaa8904 2024-04-25T14:59:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6a40a3cc0 msgr2=0x7ff6a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6a40a3cc0 0x7ff6a40a6180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009230 tx=0x7ff6b00314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 msgr2=0x7ff6c019a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 0x7ff6c019a320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00ea60 tx=0x7ff6ac00ee60 comp rx=0 tx=0).stop 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 shutdown_connections 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ff6a40a3cc0 0x7ff6a40a6180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff6c0107b50 0x7ff6c01a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0105220 0x7ff6c019a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 --2- 172.21.15.94:0/3384422614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c01028f0 0x7ff6c019a320 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 >> 172.21.15.94:0/3384422614 conn(0x7ff6c00fc300 msgr2=0x7ff6c01020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 shutdown_connections 2024-04-25T14:59:53.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.045+0000 7ff6c4c00640 1 -- 172.21.15.94:0/3384422614 wait complete. 2024-04-25T14:59:53.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T14:59:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.254+0000 7f66eea00640 1 -- 172.21.15.94:0/1839452072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e81083d0 msgr2=0x7f66e81087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.254+0000 7f66eea00640 1 --2- 172.21.15.94:0/1839452072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e81083d0 0x7f66e81087f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009900 tx=0x7f66dc02a410 comp rx=0 tx=0).stop 2024-04-25T14:59:53.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.255+0000 7f66eea00640 1 -- 172.21.15.94:0/1839452072 shutdown_connections 2024-04-25T14:59:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.255+0000 7f66eea00640 1 --2- 172.21.15.94:0/1839452072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e8109f80 0x7f66e8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.255+0000 7f66eea00640 1 --2- 172.21.15.94:0/1839452072 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e81095c0 0x7f66e8109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.255+0000 7f66eea00640 1 --2- 172.21.15.94:0/1839452072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e81083d0 0x7f66e81087f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.255+0000 7f66eea00640 1 -- 172.21.15.94:0/1839452072 >> 172.21.15.94:0/1839452072 conn(0x7f66e8077390 msgr2=0x7f66e80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.255+0000 7f66eea00640 1 -- 172.21.15.94:0/1839452072 shutdown_connections 2024-04-25T14:59:53.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.255+0000 7f66eea00640 1 -- 172.21.15.94:0/1839452072 wait complete. 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 Processor -- start 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 -- start start 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e81083d0 0x7f66e819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e81095c0 0x7f66e819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e8109f80 0x7f66e81a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8113f60 con 0x7f66e8109f80 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f66e8113de0 con 0x7f66e81095c0 2024-04-25T14:59:53.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66eea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66e81140e0 con 0x7f66e81083d0 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66ee000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e8109f80 0x7f66e81a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66ed600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e81083d0 0x7f66e819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.256+0000 7f66ee000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e8109f80 0x7f66e81a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35136/0 (socket says 172.21.15.94:35136) 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ee000640 1 -- 172.21.15.94:0/4061086957 learned_addr learned my addr 172.21.15.94:0/4061086957 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ecc00640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e81095c0 0x7f66e819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ee000640 1 -- 172.21.15.94:0/4061086957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e81083d0 msgr2=0x7f66e819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ee000640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e81083d0 0x7f66e819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ee000640 1 -- 172.21.15.94:0/4061086957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e81095c0 msgr2=0x7f66e819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ee000640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e81095c0 0x7f66e819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ee000640 1 -- 172.21.15.94:0/4061086957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e81a6210 con 0x7f66e8109f80 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66ee000640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e8109f80 0x7f66e81a5ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66e000eb10 tx=0x7f66e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.257+0000 7f66e6000640 1 -- 172.21.15.94:0/4061086957 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0004b20 con 0x7f66e8109f80 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.258+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e81a6500 con 0x7f66e8109f80 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.258+0000 7f66e6000640 1 -- 172.21.15.94:0/4061086957 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66e0004cc0 con 0x7f66e8109f80 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.258+0000 7f66e6000640 1 -- 172.21.15.94:0/4061086957 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0010690 con 0x7f66e8109f80 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.258+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e81a6a50 con 0x7f66e8109f80 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.259+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e810f6f0 con 0x7f66e8109f80 2024-04-25T14:59:53.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.261+0000 7f66e6000640 1 -- 172.21.15.94:0/4061086957 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f66e0005020 con 0x7f66e8109f80 2024-04-25T14:59:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.263+0000 7f66e6000640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66d00a3c40 0x7f66d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.263+0000 7f66e6000640 1 -- 172.21.15.94:0/4061086957 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f66e00d0a20 con 0x7f66e8109f80 2024-04-25T14:59:53.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.263+0000 7f66ed600640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66d00a3c40 0x7f66d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.264+0000 7f66ed600640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66d00a3c40 0x7f66d00a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66dc005ab0 tx=0x7f66dc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:53.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.268+0000 7f66e6000640 1 -- 172.21.15.94:0/4061086957 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66e009d350 con 0x7f66e8109f80 2024-04-25T14:59:53.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.508+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66e8062b50 con 0x7f66e8109f80 2024-04-25T14:59:53.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.510+0000 7f66e6000640 1 -- 172.21.15.94:0/4061086957 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f66e009cc50 con 0x7f66e8109f80 2024-04-25T14:59:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:53.510 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T14:59:53.511 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T14:59:53.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66d00a3c40 msgr2=0x7f66d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66d00a3c40 0x7f66d00a6100 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f66dc005ab0 tx=0x7f66dc005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e8109f80 msgr2=0x7f66e81a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e8109f80 0x7f66e81a5ba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f66e000eb10 tx=0x7f66e000c2d0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 shutdown_connections 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66d00a3c40 0x7f66d00a6100 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66e8109f80 0x7f66e81a5ba0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66e81095c0 0x7f66e819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 --2- 172.21.15.94:0/4061086957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66e81083d0 0x7f66e819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.512+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 >> 172.21.15.94:0/4061086957 conn(0x7f66e8077390 msgr2=0x7f66e810c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:53.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.513+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 shutdown_connections 2024-04-25T14:59:53.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.513+0000 7f66eea00640 1 -- 172.21.15.94:0/4061086957 wait complete. 2024-04-25T14:59:53.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 -- 172.21.15.94:0/1130066742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1083d0 msgr2=0x7f60ac1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 --2- 172.21.15.94:0/1130066742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1083d0 0x7f60ac1087f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f60a0009900 tx=0x7f60a002a3f0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 -- 172.21.15.94:0/1130066742 shutdown_connections 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 --2- 172.21.15.94:0/1130066742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60ac109f80 0x7f60ac110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 --2- 172.21.15.94:0/1130066742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60ac1095c0 0x7f60ac109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 --2- 172.21.15.94:0/1130066742 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1083d0 0x7f60ac1087f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 -- 172.21.15.94:0/1130066742 >> 172.21.15.94:0/1130066742 conn(0x7f60ac077390 msgr2=0x7f60ac0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 -- 172.21.15.94:0/1130066742 shutdown_connections 2024-04-25T14:59:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.728+0000 7f60b2800640 1 -- 172.21.15.94:0/1130066742 wait complete. 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.729+0000 7f60b2800640 1 Processor -- start 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.729+0000 7f60b2800640 1 -- start start 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60ac1083d0 0x7f60ac19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b2800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1095c0 0x7f60ac19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b2800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60ac109f80 0x7f60ac1a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b2800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac113f60 con 0x7f60ac109f80 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b2800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac113de0 con 0x7f60ac1095c0 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1140e0 con 0x7f60ac1083d0 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60ac109f80 0x7f60ac1a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60ac1083d0 0x7f60ac19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60ac1083d0 0x7f60ac19e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:42666/0 (socket says 172.21.15.94:42666) 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b0a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1095c0 0x7f60ac19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b1400640 1 -- 172.21.15.94:0/2388703996 learned_addr learned my addr 172.21.15.94:0/2388703996 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b1e00640 1 -- 172.21.15.94:0/2388703996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60ac1083d0 msgr2=0x7f60ac19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.730+0000 7f60b1e00640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60ac1083d0 0x7f60ac19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60b1e00640 1 -- 172.21.15.94:0/2388703996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1095c0 msgr2=0x7f60ac19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60b1e00640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1095c0 0x7f60ac19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60b1e00640 1 -- 172.21.15.94:0/2388703996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac1a6230 con 0x7f60ac109f80 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60b1e00640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60ac109f80 0x7f60ac1a5bc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009fd0 tx=0x7f60a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60aa000640 1 -- 172.21.15.94:0/2388703996 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a400ce20 con 0x7f60ac109f80 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60aa000640 1 -- 172.21.15.94:0/2388703996 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60a4005020 con 0x7f60ac109f80 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1a6520 con 0x7f60ac109f80 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60aa000640 1 -- 172.21.15.94:0/2388703996 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4010690 con 0x7f60ac109f80 2024-04-25T14:59:53.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.731+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1a6a20 con 0x7f60ac109f80 2024-04-25T14:59:53.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.732+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ac10f790 con 0x7f60ac109f80 2024-04-25T14:59:53.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.734+0000 7f60aa000640 1 -- 172.21.15.94:0/2388703996 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f60a40054d0 con 0x7f60ac109f80 2024-04-25T14:59:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.736+0000 7f60aa000640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60940a3c40 0x7f60940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.736+0000 7f60aa000640 1 -- 172.21.15.94:0/2388703996 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f60a40d0aa0 con 0x7f60ac109f80 2024-04-25T14:59:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.736+0000 7f60b1400640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60940a3c40 0x7f60940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:53.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.737+0000 7f60b1400640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60940a3c40 0x7f60940a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60a0000c00 tx=0x7f60a0005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:53.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.740+0000 7f60aa000640 1 -- 172.21.15.94:0/2388703996 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60a409d4d0 con 0x7f60ac109f80 2024-04-25T14:59:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.945+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60ac062b50 con 0x7f60ac109f80 2024-04-25T14:59:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.949+0000 7f60aa000640 1 -- 172.21.15.94:0/2388703996 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22204 (secure 0 0 0) 0x7f60a409cdd0 con 0x7f60ac109f80 2024-04-25T14:59:53.950 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:53.950 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":86,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T14:59:47.283028+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T14:59:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60940a3c40 msgr2=0x7f60940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60940a3c40 0x7f60940a6100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f60a0000c00 tx=0x7f60a0005a20 comp rx=0 tx=0).stop 2024-04-25T14:59:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60ac109f80 msgr2=0x7f60ac1a5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60ac109f80 0x7f60ac1a5bc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009fd0 tx=0x7f60a400ef10 comp rx=0 tx=0).stop 2024-04-25T14:59:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 shutdown_connections 2024-04-25T14:59:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60940a3c40 0x7f60940a6100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60ac109f80 0x7f60ac1a5bc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f60ac1095c0 0x7f60ac19eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 --2- 172.21.15.94:0/2388703996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60ac1083d0 0x7f60ac19e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 >> 172.21.15.94:0/2388703996 conn(0x7f60ac077390 msgr2=0x7f60ac10c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 shutdown_connections 2024-04-25T14:59:53.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:53.952+0000 7f60b2800640 1 -- 172.21.15.94:0/2388703996 wait complete. 2024-04-25T14:59:53.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:53 smithi138 ceph-mon[32875]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:53.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:53 smithi138 ceph-mon[32875]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:53.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:53 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4061086957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:53.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:53 smithi094 ceph-mon[27862]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:53.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:53 smithi094 ceph-mon[27862]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:53.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:53 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4061086957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:53.974 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T14:59:53.974 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:59:53.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:53 smithi137 ceph-mon[32439]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:53.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:53 smithi137 ceph-mon[32439]: from='client.24688 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:53.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:53 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4061086957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T14:59:54.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.176+0000 7f36d3e00640 1 -- 172.21.15.94:0/52711767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41083d0 msgr2=0x7f36d41087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.176+0000 7f36d3e00640 1 --2- 172.21.15.94:0/52711767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41083d0 0x7f36d41087f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009900 tx=0x7f36c402a400 comp rx=0 tx=0).stop 2024-04-25T14:59:54.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.176+0000 7f36d3e00640 1 -- 172.21.15.94:0/52711767 shutdown_connections 2024-04-25T14:59:54.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.176+0000 7f36d3e00640 1 --2- 172.21.15.94:0/52711767 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36d4109f80 0x7f36d4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.176+0000 7f36d3e00640 1 --2- 172.21.15.94:0/52711767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36d41095c0 0x7f36d4109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.176+0000 7f36d3e00640 1 --2- 172.21.15.94:0/52711767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41083d0 0x7f36d41087f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.176+0000 7f36d3e00640 1 -- 172.21.15.94:0/52711767 >> 172.21.15.94:0/52711767 conn(0x7f36d4077390 msgr2=0x7f36d40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:54.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.177+0000 7f36d3e00640 1 -- 172.21.15.94:0/52711767 shutdown_connections 2024-04-25T14:59:54.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.177+0000 7f36d3e00640 1 -- 172.21.15.94:0/52711767 wait complete. 2024-04-25T14:59:54.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 Processor -- start 2024-04-25T14:59:54.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 -- start start 2024-04-25T14:59:54.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36d41083d0 0x7f36d419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41095c0 0x7f36d419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36d4109f80 0x7f36d41a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4113ee0 con 0x7f36d41095c0 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4113d60 con 0x7f36d4109f80 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36d4114060 con 0x7f36d41083d0 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36ca000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41095c0 0x7f36d419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36d4109f80 0x7f36d41a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36d4109f80 0x7f36d41a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55188/0 (socket says 172.21.15.94:55188) 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36ca000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41095c0 0x7f36d419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35168/0 (socket says 172.21.15.94:35168) 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d3400640 1 -- 172.21.15.94:0/4148721951 learned_addr learned my addr 172.21.15.94:0/4148721951 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.178+0000 7f36d2a00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36d41083d0 0x7f36d419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36ca000640 1 -- 172.21.15.94:0/4148721951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36d41083d0 msgr2=0x7f36d419e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36ca000640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36d41083d0 0x7f36d419e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36ca000640 1 -- 172.21.15.94:0/4148721951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36d4109f80 msgr2=0x7f36d41a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36ca000640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36d4109f80 0x7f36d41a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36ca000640 1 -- 172.21.15.94:0/4148721951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d41a6250 con 0x7f36d41095c0 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36ca000640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41095c0 0x7f36d419eb70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00d6d0 tx=0x7f36bc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36d8a00640 1 -- 172.21.15.94:0/4148721951 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc017070 con 0x7f36d41095c0 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36d8a00640 1 -- 172.21.15.94:0/4148721951 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36bc009820 con 0x7f36d41095c0 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d41a6540 con 0x7f36d41095c0 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36d8a00640 1 -- 172.21.15.94:0/4148721951 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc004c40 con 0x7f36d41095c0 2024-04-25T14:59:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.179+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d41a6ae0 con 0x7f36d41095c0 2024-04-25T14:59:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.181+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d410f7c0 con 0x7f36d41095c0 2024-04-25T14:59:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.182+0000 7f36d8a00640 1 -- 172.21.15.94:0/4148721951 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f36bc005220 con 0x7f36d41095c0 2024-04-25T14:59:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.184+0000 7f36d8a00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f36b80a3cc0 0x7f36b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.184+0000 7f36d8a00640 1 -- 172.21.15.94:0/4148721951 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f36bc0cfea0 con 0x7f36d41095c0 2024-04-25T14:59:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.184+0000 7f36d2a00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f36b80a3cc0 0x7f36b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.184+0000 7f36d2a00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f36b80a3cc0 0x7f36b80a6180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009230 tx=0x7f36c40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.188+0000 7f36d8a00640 1 -- 172.21.15.94:0/4148721951 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36bc09c180 con 0x7f36d41095c0 2024-04-25T14:59:54.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.393+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36d4062390 con 0x7f36b80a3cc0 2024-04-25T14:59:54.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.418+0000 7f36d8a00640 1 -- 172.21.15.94:0/4148721951 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258655 (secure 0 0 0) 0x7f36d4062390 con 0x7f36b80a3cc0 2024-04-25T14:59:54.423 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:54.425 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.424+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f36b80a3cc0 msgr2=0x7f36b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.424+0000 7f36d3e00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f36b80a3cc0 0x7f36b80a6180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009230 tx=0x7f36c40314a0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41095c0 msgr2=0x7f36d419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41095c0 0x7f36d419eb70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00d6d0 tx=0x7f36bc00db90 comp rx=0 tx=0).stop 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 shutdown_connections 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f36b80a3cc0 0x7f36b80a6180 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f36d4109f80 0x7f36d41a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f36d41095c0 0x7f36d419eb70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 --2- 172.21.15.94:0/4148721951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36d41083d0 0x7f36d419e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 >> 172.21.15.94:0/4148721951 conn(0x7f36d4077390 msgr2=0x7f36d410c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:54.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 shutdown_connections 2024-04-25T14:59:54.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.425+0000 7f36d3e00640 1 -- 172.21.15.94:0/4148721951 wait complete. 2024-04-25T14:59:54.445 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-04-25T14:59:53.475335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599378,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":147,"num_read_kb":128,"num_write":229,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":259,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378500,"kb_used_data":12640,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563324,"statfs":{"total":1151940427776,"available":1151552843776,"internally_reserved":0,"allocated":12943360,"data_stored":7083798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9010,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":48,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003103"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208436+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T14:59:48.208436+0000","last_peered":"2024-04-25T14:59:48.208436+0000","last_clean":"2024-04-25T14:59:48.208436+0000","last_became_active":"2024-04-25T14:59:46.311856+0000","last_became_peered":"2024-04-25T14:59:46.311856+0000","last_unstale":"2024-04-25T14:59:48.208436+0000","last_undegraded":"2024-04-25T14:59:48.208436+0000","last_fullsized":"2024-04-25T14:59:48.208436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:07.469645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529013+0000","last_change":"2024-04-25T14:59:46.310292+0000","last_active":"2024-04-25T14:59:47.529013+0000","last_peered":"2024-04-25T14:59:47.529013+0000","last_clean":"2024-04-25T14:59:47.529013+0000","last_became_active":"2024-04-25T14:59:46.309812+0000","last_became_peered":"2024-04-25T14:59:46.309812+0000","last_unstale":"2024-04-25T14:59:47.529013+0000","last_undegraded":"2024-04-25T14:59:47.529013+0000","last_fullsized":"2024-04-25T14:59:47.529013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:52.448406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529151+0000","last_change":"2024-04-25T14:59:46.300987+0000","last_active":"2024-04-25T14:59:47.529151+0000","last_peered":"2024-04-25T14:59:47.529151+0000","last_clean":"2024-04-25T14:59:47.529151+0000","last_became_active":"2024-04-25T14:59:46.299866+0000","last_became_peered":"2024-04-25T14:59:46.299866+0000","last_unstale":"2024-04-25T14:59:47.529151+0000","last_undegraded":"2024-04-25T14:59:47.529151+0000","last_fullsized":"2024-04-25T14:59:47.529151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:09.831131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289276+0000","last_change":"2024-04-25T14:59:46.304278+0000","last_active":"2024-04-25T14:59:47.289276+0000","last_peered":"2024-04-25T14:59:47.289276+0000","last_clean":"2024-04-25T14:59:47.289276+0000","last_became_active":"2024-04-25T14:59:46.303956+0000","last_became_peered":"2024-04-25T14:59:46.303956+0000","last_unstale":"2024-04-25T14:59:47.289276+0000","last_undegraded":"2024-04-25T14:59:47.289276+0000","last_fullsized":"2024-04-25T14:59:47.289276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.752548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.580955+0000","last_change":"2024-04-25T14:59:46.323730+0000","last_active":"2024-04-25T14:59:47.580955+0000","last_peered":"2024-04-25T14:59:47.580955+0000","last_clean":"2024-04-25T14:59:47.580955+0000","last_became_active":"2024-04-25T14:59:46.323383+0000","last_became_peered":"2024-04-25T14:59:46.323383+0000","last_unstale":"2024-04-25T14:59:47.580955+0000","last_undegraded":"2024-04-25T14:59:47.580955+0000","last_fullsized":"2024-04-25T14:59:47.580955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.846392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592911+0000","last_change":"2024-04-25T14:59:46.317623+0000","last_active":"2024-04-25T14:59:47.592911+0000","last_peered":"2024-04-25T14:59:47.592911+0000","last_clean":"2024-04-25T14:59:47.592911+0000","last_became_active":"2024-04-25T14:59:46.317255+0000","last_became_peered":"2024-04-25T14:59:46.317255+0000","last_unstale":"2024-04-25T14:59:47.592911+0000","last_undegraded":"2024-04-25T14:59:47.592911+0000","last_fullsized":"2024-04-25T14:59:47.592911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:24.449615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592442+0000","last_change":"2024-04-25T14:59:46.318553+0000","last_active":"2024-04-25T14:59:47.592442+0000","last_peered":"2024-04-25T14:59:47.592442+0000","last_clean":"2024-04-25T14:59:47.592442+0000","last_became_active":"2024-04-25T14:59:46.318070+0000","last_became_peered":"2024-04-25T14:59:46.318070+0000","last_unstale":"2024-04-25T14:59:47.592442+0000","last_undegraded":"2024-04-25T14:59:47.592442+0000","last_fullsized":"2024-04-25T14:59:47.592442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:41.906324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291126+0000","last_change":"2024-04-25T14:59:46.327687+0000","last_active":"2024-04-25T14:59:47.291126+0000","last_peered":"2024-04-25T14:59:47.291126+0000","last_clean":"2024-04-25T14:59:47.291126+0000","last_became_active":"2024-04-25T14:59:46.325905+0000","last_became_peered":"2024-04-25T14:59:46.325905+0000","last_unstale":"2024-04-25T14:59:47.291126+0000","last_undegraded":"2024-04-25T14:59:47.291126+0000","last_fullsized":"2024-04-25T14:59:47.291126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:50.058801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592853+0000","last_change":"2024-04-25T14:59:46.304722+0000","last_active":"2024-04-25T14:59:47.592853+0000","last_peered":"2024-04-25T14:59:47.592853+0000","last_clean":"2024-04-25T14:59:47.592853+0000","last_became_active":"2024-04-25T14:59:46.304391+0000","last_became_peered":"2024-04-25T14:59:46.304391+0000","last_unstale":"2024-04-25T14:59:47.592853+0000","last_undegraded":"2024-04-25T14:59:47.592853+0000","last_fullsized":"2024-04-25T14:59:47.592853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:15.735670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593615+0000","last_change":"2024-04-25T14:59:46.307796+0000","last_active":"2024-04-25T14:59:47.593615+0000","last_peered":"2024-04-25T14:59:47.593615+0000","last_clean":"2024-04-25T14:59:47.593615+0000","last_became_active":"2024-04-25T14:59:46.306741+0000","last_became_peered":"2024-04-25T14:59:46.306741+0000","last_unstale":"2024-04-25T14:59:47.593615+0000","last_undegraded":"2024-04-25T14:59:47.593615+0000","last_fullsized":"2024-04-25T14:59:47.593615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:34.407272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593401+0000","last_change":"2024-04-25T14:59:46.306390+0000","last_active":"2024-04-25T14:59:47.593401+0000","last_peered":"2024-04-25T14:59:47.593401+0000","last_clean":"2024-04-25T14:59:47.593401+0000","last_became_active":"2024-04-25T14:59:46.306026+0000","last_became_peered":"2024-04-25T14:59:46.306026+0000","last_unstale":"2024-04-25T14:59:47.593401+0000","last_undegraded":"2024-04-25T14:59:47.593401+0000","last_fullsized":"2024-04-25T14:59:47.593401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:59.340382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289247+0000","last_change":"2024-04-25T14:59:46.305840+0000","last_active":"2024-04-25T14:59:47.289247+0000","last_peered":"2024-04-25T14:59:47.289247+0000","last_clean":"2024-04-25T14:59:47.289247+0000","last_became_active":"2024-04-25T14:59:46.304867+0000","last_became_peered":"2024-04-25T14:59:46.304867+0000","last_unstale":"2024-04-25T14:59:47.289247+0000","last_undegraded":"2024-04-25T14:59:47.289247+0000","last_fullsized":"2024-04-25T14:59:47.289247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:58.569011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594966+0000","last_change":"2024-04-25T14:59:46.318044+0000","last_active":"2024-04-25T14:59:47.594966+0000","last_peered":"2024-04-25T14:59:47.594966+0000","last_clean":"2024-04-25T14:59:47.594966+0000","last_became_active":"2024-04-25T14:59:46.315935+0000","last_became_peered":"2024-04-25T14:59:46.315935+0000","last_unstale":"2024-04-25T14:59:47.594966+0000","last_undegraded":"2024-04-25T14:59:47.594966+0000","last_fullsized":"2024-04-25T14:59:47.594966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:42.366802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.528670+0000","last_change":"2024-04-25T14:59:46.308565+0000","last_active":"2024-04-25T14:59:47.528670+0000","last_peered":"2024-04-25T14:59:47.528670+0000","last_clean":"2024-04-25T14:59:47.528670+0000","last_became_active":"2024-04-25T14:59:46.306198+0000","last_became_peered":"2024-04-25T14:59:46.306198+0000","last_unstale":"2024-04-25T14:59:47.528670+0000","last_undegraded":"2024-04-25T14:59:47.528670+0000","last_fullsized":"2024-04-25T14:59:47.528670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:12.686162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.528490+0000","last_change":"2024-04-25T14:59:46.308313+0000","last_active":"2024-04-25T14:59:47.528490+0000","last_peered":"2024-04-25T14:59:47.528490+0000","last_clean":"2024-04-25T14:59:47.528490+0000","last_became_active":"2024-04-25T14:59:46.305974+0000","last_became_peered":"2024-04-25T14:59:46.305974+0000","last_unstale":"2024-04-25T14:59:47.528490+0000","last_undegraded":"2024-04-25T14:59:47.528490+0000","last_fullsized":"2024-04-25T14:59:47.528490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.841983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591641+0000","last_change":"2024-04-25T14:59:46.302726+0000","last_active":"2024-04-25T14:59:47.591641+0000","last_peered":"2024-04-25T14:59:47.591641+0000","last_clean":"2024-04-25T14:59:47.591641+0000","last_became_active":"2024-04-25T14:59:46.302406+0000","last_became_peered":"2024-04-25T14:59:46.302406+0000","last_unstale":"2024-04-25T14:59:47.591641+0000","last_undegraded":"2024-04-25T14:59:47.591641+0000","last_fullsized":"2024-04-25T14:59:47.591641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208554+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T14:59:48.208554+0000","last_peered":"2024-04-25T14:59:48.208554+0000","last_clean":"2024-04-25T14:59:48.208554+0000","last_became_active":"2024-04-25T14:59:46.311822+0000","last_became_peered":"2024-04-25T14:59:46.311822+0000","last_unstale":"2024-04-25T14:59:48.208554+0000","last_undegraded":"2024-04-25T14:59:48.208554+0000","last_fullsized":"2024-04-25T14:59:48.208554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.128537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578371+0000","last_change":"2024-04-25T14:59:46.313816+0000","last_active":"2024-04-25T14:59:47.578371+0000","last_peered":"2024-04-25T14:59:47.578371+0000","last_clean":"2024-04-25T14:59:47.578371+0000","last_became_active":"2024-04-25T14:59:46.313462+0000","last_became_peered":"2024-04-25T14:59:46.313462+0000","last_unstale":"2024-04-25T14:59:47.578371+0000","last_undegraded":"2024-04-25T14:59:47.578371+0000","last_fullsized":"2024-04-25T14:59:47.578371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:29.402113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592037+0000","last_change":"2024-04-25T14:59:46.317972+0000","last_active":"2024-04-25T14:59:47.592037+0000","last_peered":"2024-04-25T14:59:47.592037+0000","last_clean":"2024-04-25T14:59:47.592037+0000","last_became_active":"2024-04-25T14:59:46.317466+0000","last_became_peered":"2024-04-25T14:59:46.317466+0000","last_unstale":"2024-04-25T14:59:47.592037+0000","last_undegraded":"2024-04-25T14:59:47.592037+0000","last_fullsized":"2024-04-25T14:59:47.592037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:03.706663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578572+0000","last_change":"2024-04-25T14:59:45.320752+0000","last_active":"2024-04-25T14:59:47.578572+0000","last_peered":"2024-04-25T14:59:47.578572+0000","last_clean":"2024-04-25T14:59:47.578572+0000","last_became_active":"2024-04-25T14:59:45.319463+0000","last_became_peered":"2024-04-25T14:59:45.319463+0000","last_unstale":"2024-04-25T14:59:47.578572+0000","last_undegraded":"2024-04-25T14:59:47.578572+0000","last_fullsized":"2024-04-25T14:59:47.578572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:52.608899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.292922+0000","last_change":"2024-04-25T14:59:46.323802+0000","last_active":"2024-04-25T14:59:47.292922+0000","last_peered":"2024-04-25T14:59:47.292922+0000","last_clean":"2024-04-25T14:59:47.292922+0000","last_became_active":"2024-04-25T14:59:46.323498+0000","last_became_peered":"2024-04-25T14:59:46.323498+0000","last_unstale":"2024-04-25T14:59:47.292922+0000","last_undegraded":"2024-04-25T14:59:47.292922+0000","last_fullsized":"2024-04-25T14:59:47.292922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:38.236936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.292883+0000","last_change":"2024-04-25T14:59:45.335156+0000","last_active":"2024-04-25T14:59:47.292883+0000","last_peered":"2024-04-25T14:59:47.292883+0000","last_clean":"2024-04-25T14:59:47.292883+0000","last_became_active":"2024-04-25T14:59:45.333589+0000","last_became_peered":"2024-04-25T14:59:45.333589+0000","last_unstale":"2024-04-25T14:59:47.292883+0000","last_undegraded":"2024-04-25T14:59:47.292883+0000","last_fullsized":"2024-04-25T14:59:47.292883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529610+0000","last_change":"2024-04-25T14:59:46.319165+0000","last_active":"2024-04-25T14:59:47.529610+0000","last_peered":"2024-04-25T14:59:47.529610+0000","last_clean":"2024-04-25T14:59:47.529610+0000","last_became_active":"2024-04-25T14:59:46.318801+0000","last_became_peered":"2024-04-25T14:59:46.318801+0000","last_unstale":"2024-04-25T14:59:47.529610+0000","last_undegraded":"2024-04-25T14:59:47.529610+0000","last_fullsized":"2024-04-25T14:59:47.529610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:01.806099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581387+0000","last_change":"2024-04-25T14:59:45.317956+0000","last_active":"2024-04-25T14:59:47.581387+0000","last_peered":"2024-04-25T14:59:47.581387+0000","last_clean":"2024-04-25T14:59:47.581387+0000","last_became_active":"2024-04-25T14:59:45.317655+0000","last_became_peered":"2024-04-25T14:59:45.317655+0000","last_unstale":"2024-04-25T14:59:47.581387+0000","last_undegraded":"2024-04-25T14:59:47.581387+0000","last_fullsized":"2024-04-25T14:59:47.581387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:59.008411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592332+0000","last_change":"2024-04-25T14:59:46.311659+0000","last_active":"2024-04-25T14:59:47.592332+0000","last_peered":"2024-04-25T14:59:47.592332+0000","last_clean":"2024-04-25T14:59:47.592332+0000","last_became_active":"2024-04-25T14:59:46.311287+0000","last_became_peered":"2024-04-25T14:59:46.311287+0000","last_unstale":"2024-04-25T14:59:47.592332+0000","last_undegraded":"2024-04-25T14:59:47.592332+0000","last_fullsized":"2024-04-25T14:59:47.592332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:29.434223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581133+0000","last_change":"2024-04-25T14:59:45.317658+0000","last_active":"2024-04-25T14:59:47.581133+0000","last_peered":"2024-04-25T14:59:47.581133+0000","last_clean":"2024-04-25T14:59:47.581133+0000","last_became_active":"2024-04-25T14:59:45.317311+0000","last_became_peered":"2024-04-25T14:59:45.317311+0000","last_unstale":"2024-04-25T14:59:47.581133+0000","last_undegraded":"2024-04-25T14:59:47.581133+0000","last_fullsized":"2024-04-25T14:59:47.581133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.602412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291939+0000","last_change":"2024-04-25T14:59:46.305456+0000","last_active":"2024-04-25T14:59:47.291939+0000","last_peered":"2024-04-25T14:59:47.291939+0000","last_clean":"2024-04-25T14:59:47.291939+0000","last_became_active":"2024-04-25T14:59:46.305143+0000","last_became_peered":"2024-04-25T14:59:46.305143+0000","last_unstale":"2024-04-25T14:59:47.291939+0000","last_undegraded":"2024-04-25T14:59:47.291939+0000","last_fullsized":"2024-04-25T14:59:47.291939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:03.941200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291489+0000","last_change":"2024-04-25T14:59:45.315407+0000","last_active":"2024-04-25T14:59:47.291489+0000","last_peered":"2024-04-25T14:59:47.291489+0000","last_clean":"2024-04-25T14:59:47.291489+0000","last_became_active":"2024-04-25T14:59:45.315070+0000","last_became_peered":"2024-04-25T14:59:45.315070+0000","last_unstale":"2024-04-25T14:59:47.291489+0000","last_undegraded":"2024-04-25T14:59:47.291489+0000","last_fullsized":"2024-04-25T14:59:47.291489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:32.492027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581007+0000","last_change":"2024-04-25T14:59:46.318218+0000","last_active":"2024-04-25T14:59:47.581007+0000","last_peered":"2024-04-25T14:59:47.581007+0000","last_clean":"2024-04-25T14:59:47.581007+0000","last_became_active":"2024-04-25T14:59:46.317855+0000","last_became_peered":"2024-04-25T14:59:46.317855+0000","last_unstale":"2024-04-25T14:59:47.581007+0000","last_undegraded":"2024-04-25T14:59:47.581007+0000","last_fullsized":"2024-04-25T14:59:47.581007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:11.712691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592057+0000","last_change":"2024-04-25T14:59:45.319624+0000","last_active":"2024-04-25T14:59:47.592057+0000","last_peered":"2024-04-25T14:59:47.592057+0000","last_clean":"2024-04-25T14:59:47.592057+0000","last_became_active":"2024-04-25T14:59:45.319310+0000","last_became_peered":"2024-04-25T14:59:45.319310+0000","last_unstale":"2024-04-25T14:59:47.592057+0000","last_undegraded":"2024-04-25T14:59:47.592057+0000","last_fullsized":"2024-04-25T14:59:47.592057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:40.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291556+0000","last_change":"2024-04-25T14:59:46.317382+0000","last_active":"2024-04-25T14:59:47.291556+0000","last_peered":"2024-04-25T14:59:47.291556+0000","last_clean":"2024-04-25T14:59:47.291556+0000","last_became_active":"2024-04-25T14:59:46.316993+0000","last_became_peered":"2024-04-25T14:59:46.316993+0000","last_unstale":"2024-04-25T14:59:47.291556+0000","last_undegraded":"2024-04-25T14:59:47.291556+0000","last_fullsized":"2024-04-25T14:59:47.291556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:06.618964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593927+0000","last_change":"2024-04-25T14:59:45.322731+0000","last_active":"2024-04-25T14:59:47.593927+0000","last_peered":"2024-04-25T14:59:47.593927+0000","last_clean":"2024-04-25T14:59:47.593927+0000","last_became_active":"2024-04-25T14:59:45.322360+0000","last_became_peered":"2024-04-25T14:59:45.322360+0000","last_unstale":"2024-04-25T14:59:47.593927+0000","last_undegraded":"2024-04-25T14:59:47.593927+0000","last_fullsized":"2024-04-25T14:59:47.593927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:02.890016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291785+0000","last_change":"2024-04-25T14:59:46.327195+0000","last_active":"2024-04-25T14:59:47.291785+0000","last_peered":"2024-04-25T14:59:47.291785+0000","last_clean":"2024-04-25T14:59:47.291785+0000","last_became_active":"2024-04-25T14:59:46.322053+0000","last_became_peered":"2024-04-25T14:59:46.322053+0000","last_unstale":"2024-04-25T14:59:47.291785+0000","last_undegraded":"2024-04-25T14:59:47.291785+0000","last_fullsized":"2024-04-25T14:59:47.291785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:35.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291231+0000","last_change":"2024-04-25T14:59:45.315688+0000","last_active":"2024-04-25T14:59:47.291231+0000","last_peered":"2024-04-25T14:59:47.291231+0000","last_clean":"2024-04-25T14:59:47.291231+0000","last_became_active":"2024-04-25T14:59:45.315074+0000","last_became_peered":"2024-04-25T14:59:45.315074+0000","last_unstale":"2024-04-25T14:59:47.291231+0000","last_undegraded":"2024-04-25T14:59:47.291231+0000","last_fullsized":"2024-04-25T14:59:47.291231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:49.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.577841+0000","last_change":"2024-04-25T14:59:46.322874+0000","last_active":"2024-04-25T14:59:47.577841+0000","last_peered":"2024-04-25T14:59:47.577841+0000","last_clean":"2024-04-25T14:59:47.577841+0000","last_became_active":"2024-04-25T14:59:46.322492+0000","last_became_peered":"2024-04-25T14:59:46.322492+0000","last_unstale":"2024-04-25T14:59:47.577841+0000","last_undegraded":"2024-04-25T14:59:47.577841+0000","last_fullsized":"2024-04-25T14:59:47.577841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:21.294792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593230+0000","last_change":"2024-04-25T14:59:45.322448+0000","last_active":"2024-04-25T14:59:47.593230+0000","last_peered":"2024-04-25T14:59:47.593230+0000","last_clean":"2024-04-25T14:59:47.593230+0000","last_became_active":"2024-04-25T14:59:45.322127+0000","last_became_peered":"2024-04-25T14:59:45.322127+0000","last_unstale":"2024-04-25T14:59:47.593230+0000","last_undegraded":"2024-04-25T14:59:47.593230+0000","last_fullsized":"2024-04-25T14:59:47.593230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.284668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592432+0000","last_change":"2024-04-25T14:59:46.318371+0000","last_active":"2024-04-25T14:59:47.592432+0000","last_peered":"2024-04-25T14:59:47.592432+0000","last_clean":"2024-04-25T14:59:47.592432+0000","last_became_active":"2024-04-25T14:59:46.317819+0000","last_became_peered":"2024-04-25T14:59:46.317819+0000","last_unstale":"2024-04-25T14:59:47.592432+0000","last_undegraded":"2024-04-25T14:59:47.592432+0000","last_fullsized":"2024-04-25T14:59:47.592432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.047982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592846+0000","last_change":"2024-04-25T14:59:45.318286+0000","last_active":"2024-04-25T14:59:47.592846+0000","last_peered":"2024-04-25T14:59:47.592846+0000","last_clean":"2024-04-25T14:59:47.592846+0000","last_became_active":"2024-04-25T14:59:45.317787+0000","last_became_peered":"2024-04-25T14:59:45.317787+0000","last_unstale":"2024-04-25T14:59:47.592846+0000","last_undegraded":"2024-04-25T14:59:47.592846+0000","last_fullsized":"2024-04-25T14:59:47.592846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:06.508164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529494+0000","last_change":"2024-04-25T14:59:46.309331+0000","last_active":"2024-04-25T14:59:47.529494+0000","last_peered":"2024-04-25T14:59:47.529494+0000","last_clean":"2024-04-25T14:59:47.529494+0000","last_became_active":"2024-04-25T14:59:46.306252+0000","last_became_peered":"2024-04-25T14:59:46.306252+0000","last_unstale":"2024-04-25T14:59:47.529494+0000","last_undegraded":"2024-04-25T14:59:47.529494+0000","last_fullsized":"2024-04-25T14:59:47.529494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:53.896138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291834+0000","last_change":"2024-04-25T14:59:45.327951+0000","last_active":"2024-04-25T14:59:47.291834+0000","last_peered":"2024-04-25T14:59:47.291834+0000","last_clean":"2024-04-25T14:59:47.291834+0000","last_became_active":"2024-04-25T14:59:45.327660+0000","last_became_peered":"2024-04-25T14:59:45.327660+0000","last_unstale":"2024-04-25T14:59:47.291834+0000","last_undegraded":"2024-04-25T14:59:47.291834+0000","last_fullsized":"2024-04-25T14:59:47.291834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:29.862798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592221+0000","last_change":"2024-04-25T14:59:46.304038+0000","last_active":"2024-04-25T14:59:47.592221+0000","last_peered":"2024-04-25T14:59:47.592221+0000","last_clean":"2024-04-25T14:59:47.592221+0000","last_became_active":"2024-04-25T14:59:46.303713+0000","last_became_peered":"2024-04-25T14:59:46.303713+0000","last_unstale":"2024-04-25T14:59:47.592221+0000","last_undegraded":"2024-04-25T14:59:47.592221+0000","last_fullsized":"2024-04-25T14:59:47.592221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:30.476591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:51.126826+0000","last_change":"2024-04-25T14:59:45.315738+0000","last_active":"2024-04-25T14:59:51.126826+0000","last_peered":"2024-04-25T14:59:51.126826+0000","last_clean":"2024-04-25T14:59:51.126826+0000","last_became_active":"2024-04-25T14:59:45.315423+0000","last_became_peered":"2024-04-25T14:59:45.315423+0000","last_unstale":"2024-04-25T14:59:51.126826+0000","last_undegraded":"2024-04-25T14:59:51.126826+0000","last_fullsized":"2024-04-25T14:59:51.126826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:33.135348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581304+0000","last_change":"2024-04-25T14:59:46.308085+0000","last_active":"2024-04-25T14:59:47.581304+0000","last_peered":"2024-04-25T14:59:47.581304+0000","last_clean":"2024-04-25T14:59:47.581304+0000","last_became_active":"2024-04-25T14:59:46.307739+0000","last_became_peered":"2024-04-25T14:59:46.307739+0000","last_unstale":"2024-04-25T14:59:47.581304+0000","last_undegraded":"2024-04-25T14:59:47.581304+0000","last_fullsized":"2024-04-25T14:59:47.581304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:57.949147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591667+0000","last_change":"2024-04-25T14:59:45.327058+0000","last_active":"2024-04-25T14:59:47.591667+0000","last_peered":"2024-04-25T14:59:47.591667+0000","last_clean":"2024-04-25T14:59:47.591667+0000","last_became_active":"2024-04-25T14:59:45.326817+0000","last_became_peered":"2024-04-25T14:59:45.326817+0000","last_unstale":"2024-04-25T14:59:47.591667+0000","last_undegraded":"2024-04-25T14:59:47.591667+0000","last_fullsized":"2024-04-25T14:59:47.591667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:51.087966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594493+0000","last_change":"2024-04-25T14:59:46.328901+0000","last_active":"2024-04-25T14:59:47.594493+0000","last_peered":"2024-04-25T14:59:47.594493+0000","last_clean":"2024-04-25T14:59:47.594493+0000","last_became_active":"2024-04-25T14:59:46.328496+0000","last_became_peered":"2024-04-25T14:59:46.328496+0000","last_unstale":"2024-04-25T14:59:47.594493+0000","last_undegraded":"2024-04-25T14:59:47.594493+0000","last_fullsized":"2024-04-25T14:59:47.594493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:48.149719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291330+0000","last_change":"2024-04-25T14:59:45.316673+0000","last_active":"2024-04-25T14:59:47.291330+0000","last_peered":"2024-04-25T14:59:47.291330+0000","last_clean":"2024-04-25T14:59:47.291330+0000","last_became_active":"2024-04-25T14:59:45.316280+0000","last_became_peered":"2024-04-25T14:59:45.316280+0000","last_unstale":"2024-04-25T14:59:47.291330+0000","last_undegraded":"2024-04-25T14:59:47.291330+0000","last_fullsized":"2024-04-25T14:59:47.291330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:51.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592813+0000","last_change":"2024-04-25T14:59:46.323821+0000","last_active":"2024-04-25T14:59:47.592813+0000","last_peered":"2024-04-25T14:59:47.592813+0000","last_clean":"2024-04-25T14:59:47.592813+0000","last_became_active":"2024-04-25T14:59:46.323434+0000","last_became_peered":"2024-04-25T14:59:46.323434+0000","last_unstale":"2024-04-25T14:59:47.592813+0000","last_undegraded":"2024-04-25T14:59:47.592813+0000","last_fullsized":"2024-04-25T14:59:47.592813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:20.983270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.288973+0000","last_change":"2024-04-25T14:59:45.317953+0000","last_active":"2024-04-25T14:59:47.288973+0000","last_peered":"2024-04-25T14:59:47.288973+0000","last_clean":"2024-04-25T14:59:47.288973+0000","last_became_active":"2024-04-25T14:59:45.317617+0000","last_became_peered":"2024-04-25T14:59:45.317617+0000","last_unstale":"2024-04-25T14:59:47.288973+0000","last_undegraded":"2024-04-25T14:59:47.288973+0000","last_fullsized":"2024-04-25T14:59:47.288973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:44.235492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591705+0000","last_change":"2024-04-25T14:59:46.305929+0000","last_active":"2024-04-25T14:59:47.591705+0000","last_peered":"2024-04-25T14:59:47.591705+0000","last_clean":"2024-04-25T14:59:47.591705+0000","last_became_active":"2024-04-25T14:59:46.305604+0000","last_became_peered":"2024-04-25T14:59:46.305604+0000","last_unstale":"2024-04-25T14:59:47.591705+0000","last_undegraded":"2024-04-25T14:59:47.591705+0000","last_fullsized":"2024-04-25T14:59:47.591705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:37.340009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.580741+0000","last_change":"2024-04-25T14:59:45.321385+0000","last_active":"2024-04-25T14:59:47.580741+0000","last_peered":"2024-04-25T14:59:47.580741+0000","last_clean":"2024-04-25T14:59:47.580741+0000","last_became_active":"2024-04-25T14:59:45.321057+0000","last_became_peered":"2024-04-25T14:59:45.321057+0000","last_unstale":"2024-04-25T14:59:47.580741+0000","last_undegraded":"2024-04-25T14:59:47.580741+0000","last_fullsized":"2024-04-25T14:59:47.580741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:24.907017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291274+0000","last_change":"2024-04-25T14:59:46.306259+0000","last_active":"2024-04-25T14:59:47.291274+0000","last_peered":"2024-04-25T14:59:47.291274+0000","last_clean":"2024-04-25T14:59:47.291274+0000","last_became_active":"2024-04-25T14:59:46.305938+0000","last_became_peered":"2024-04-25T14:59:46.305938+0000","last_unstale":"2024-04-25T14:59:47.291274+0000","last_undegraded":"2024-04-25T14:59:47.291274+0000","last_fullsized":"2024-04-25T14:59:47.291274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:24.931749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.985502+0000","last_change":"2024-04-25T14:59:45.318291+0000","last_active":"2024-04-25T14:59:47.985502+0000","last_peered":"2024-04-25T14:59:47.985502+0000","last_clean":"2024-04-25T14:59:47.985502+0000","last_became_active":"2024-04-25T14:59:45.317590+0000","last_became_peered":"2024-04-25T14:59:45.317590+0000","last_unstale":"2024-04-25T14:59:47.985502+0000","last_undegraded":"2024-04-25T14:59:47.985502+0000","last_fullsized":"2024-04-25T14:59:47.985502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208347+0000","last_change":"2024-04-25T14:59:46.303503+0000","last_active":"2024-04-25T14:59:48.208347+0000","last_peered":"2024-04-25T14:59:48.208347+0000","last_clean":"2024-04-25T14:59:48.208347+0000","last_became_active":"2024-04-25T14:59:46.303127+0000","last_became_peered":"2024-04-25T14:59:46.303127+0000","last_unstale":"2024-04-25T14:59:48.208347+0000","last_undegraded":"2024-04-25T14:59:48.208347+0000","last_fullsized":"2024-04-25T14:59:48.208347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:46.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578157+0000","last_change":"2024-04-25T14:59:45.330010+0000","last_active":"2024-04-25T14:59:47.578157+0000","last_peered":"2024-04-25T14:59:47.578157+0000","last_clean":"2024-04-25T14:59:47.578157+0000","last_became_active":"2024-04-25T14:59:45.329225+0000","last_became_peered":"2024-04-25T14:59:45.329225+0000","last_unstale":"2024-04-25T14:59:47.578157+0000","last_undegraded":"2024-04-25T14:59:47.578157+0000","last_fullsized":"2024-04-25T14:59:47.578157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:01.623458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593114+0000","last_change":"2024-04-25T14:59:46.320003+0000","last_active":"2024-04-25T14:59:47.593114+0000","last_peered":"2024-04-25T14:59:47.593114+0000","last_clean":"2024-04-25T14:59:47.593114+0000","last_became_active":"2024-04-25T14:59:46.319600+0000","last_became_peered":"2024-04-25T14:59:46.319600+0000","last_unstale":"2024-04-25T14:59:47.593114+0000","last_undegraded":"2024-04-25T14:59:47.593114+0000","last_fullsized":"2024-04-25T14:59:47.593114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.916018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592595+0000","last_change":"2024-04-25T14:59:45.325233+0000","last_active":"2024-04-25T14:59:47.592595+0000","last_peered":"2024-04-25T14:59:47.592595+0000","last_clean":"2024-04-25T14:59:47.592595+0000","last_became_active":"2024-04-25T14:59:45.324826+0000","last_became_peered":"2024-04-25T14:59:45.324826+0000","last_unstale":"2024-04-25T14:59:47.592595+0000","last_undegraded":"2024-04-25T14:59:47.592595+0000","last_fullsized":"2024-04-25T14:59:47.592595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.422559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593303+0000","last_change":"2024-04-25T14:59:46.314552+0000","last_active":"2024-04-25T14:59:47.593303+0000","last_peered":"2024-04-25T14:59:47.593303+0000","last_clean":"2024-04-25T14:59:47.593303+0000","last_became_active":"2024-04-25T14:59:46.314159+0000","last_became_peered":"2024-04-25T14:59:46.314159+0000","last_unstale":"2024-04-25T14:59:47.593303+0000","last_undegraded":"2024-04-25T14:59:47.593303+0000","last_fullsized":"2024-04-25T14:59:47.593303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:06.731960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291682+0000","last_change":"2024-04-25T14:59:46.303354+0000","last_active":"2024-04-25T14:59:47.291682+0000","last_peered":"2024-04-25T14:59:47.291682+0000","last_clean":"2024-04-25T14:59:47.291682+0000","last_became_active":"2024-04-25T14:59:46.302869+0000","last_became_peered":"2024-04-25T14:59:46.302869+0000","last_unstale":"2024-04-25T14:59:47.291682+0000","last_undegraded":"2024-04-25T14:59:47.291682+0000","last_fullsized":"2024-04-25T14:59:47.291682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.796902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593646+0000","last_change":"2024-04-25T14:59:45.336686+0000","last_active":"2024-04-25T14:59:47.593646+0000","last_peered":"2024-04-25T14:59:47.593646+0000","last_clean":"2024-04-25T14:59:47.593646+0000","last_became_active":"2024-04-25T14:59:45.335828+0000","last_became_peered":"2024-04-25T14:59:45.335828+0000","last_unstale":"2024-04-25T14:59:47.593646+0000","last_undegraded":"2024-04-25T14:59:47.593646+0000","last_fullsized":"2024-04-25T14:59:47.593646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.676117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291807+0000","last_change":"2024-04-25T14:59:46.319541+0000","last_active":"2024-04-25T14:59:47.291807+0000","last_peered":"2024-04-25T14:59:47.291807+0000","last_clean":"2024-04-25T14:59:47.291807+0000","last_became_active":"2024-04-25T14:59:46.317922+0000","last_became_peered":"2024-04-25T14:59:46.317922+0000","last_unstale":"2024-04-25T14:59:47.291807+0000","last_undegraded":"2024-04-25T14:59:47.291807+0000","last_fullsized":"2024-04-25T14:59:47.291807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.454075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289000+0000","last_change":"2024-04-25T14:59:45.313817+0000","last_active":"2024-04-25T14:59:47.289000+0000","last_peered":"2024-04-25T14:59:47.289000+0000","last_clean":"2024-04-25T14:59:47.289000+0000","last_became_active":"2024-04-25T14:59:45.313507+0000","last_became_peered":"2024-04-25T14:59:45.313507+0000","last_unstale":"2024-04-25T14:59:47.289000+0000","last_undegraded":"2024-04-25T14:59:47.289000+0000","last_fullsized":"2024-04-25T14:59:47.289000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:18.126351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591814+0000","last_change":"2024-04-25T14:58:29.935018+0000","last_active":"2024-04-25T14:59:47.591814+0000","last_peered":"2024-04-25T14:59:47.591814+0000","last_clean":"2024-04-25T14:59:47.591814+0000","last_became_active":"2024-04-25T14:58:29.934454+0000","last_became_peered":"2024-04-25T14:58:29.934454+0000","last_unstale":"2024-04-25T14:59:47.591814+0000","last_undegraded":"2024-04-25T14:59:47.591814+0000","last_fullsized":"2024-04-25T14:59:47.591814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_clean_scrub_stamp":"2024-04-25T14:54:46.455802+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:09.308783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593901+0000","last_change":"2024-04-25T14:59:46.319680+0000","last_active":"2024-04-25T14:59:47.593901+0000","last_peered":"2024-04-25T14:59:47.593901+0000","last_clean":"2024-04-25T14:59:47.593901+0000","last_became_active":"2024-04-25T14:59:46.319225+0000","last_became_peered":"2024-04-25T14:59:46.319225+0000","last_unstale":"2024-04-25T14:59:47.593901+0000","last_undegraded":"2024-04-25T14:59:47.593901+0000","last_fullsized":"2024-04-25T14:59:47.593901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:34.629244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592073+0000","last_change":"2024-04-25T14:59:45.318087+0000","last_active":"2024-04-25T14:59:47.592073+0000","last_peered":"2024-04-25T14:59:47.592073+0000","last_clean":"2024-04-25T14:59:47.592073+0000","last_became_active":"2024-04-25T14:59:45.317745+0000","last_became_peered":"2024-04-25T14:59:45.317745+0000","last_unstale":"2024-04-25T14:59:47.592073+0000","last_undegraded":"2024-04-25T14:59:47.592073+0000","last_fullsized":"2024-04-25T14:59:47.592073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:17.202201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581234+0000","last_change":"2024-04-25T14:59:45.332358+0000","last_active":"2024-04-25T14:59:47.581234+0000","last_peered":"2024-04-25T14:59:47.581234+0000","last_clean":"2024-04-25T14:59:47.581234+0000","last_became_active":"2024-04-25T14:59:45.332089+0000","last_became_peered":"2024-04-25T14:59:45.332089+0000","last_unstale":"2024-04-25T14:59:47.581234+0000","last_undegraded":"2024-04-25T14:59:47.581234+0000","last_fullsized":"2024-04-25T14:59:47.581234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:50.937548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594844+0000","last_change":"2024-04-25T14:59:46.320945+0000","last_active":"2024-04-25T14:59:47.594844+0000","last_peered":"2024-04-25T14:59:47.594844+0000","last_clean":"2024-04-25T14:59:47.594844+0000","last_became_active":"2024-04-25T14:59:46.320602+0000","last_became_peered":"2024-04-25T14:59:46.320602+0000","last_unstale":"2024-04-25T14:59:47.594844+0000","last_undegraded":"2024-04-25T14:59:47.594844+0000","last_fullsized":"2024-04-25T14:59:47.594844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:32.282647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578436+0000","last_change":"2024-04-25T14:59:45.334325+0000","last_active":"2024-04-25T14:59:47.578436+0000","last_peered":"2024-04-25T14:59:47.578436+0000","last_clean":"2024-04-25T14:59:47.578436+0000","last_became_active":"2024-04-25T14:59:45.333969+0000","last_became_peered":"2024-04-25T14:59:45.333969+0000","last_unstale":"2024-04-25T14:59:47.578436+0000","last_undegraded":"2024-04-25T14:59:47.578436+0000","last_fullsized":"2024-04-25T14:59:47.578436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:02.694267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291682+0000","last_change":"2024-04-25T14:59:46.323686+0000","last_active":"2024-04-25T14:59:47.291682+0000","last_peered":"2024-04-25T14:59:47.291682+0000","last_clean":"2024-04-25T14:59:47.291682+0000","last_became_active":"2024-04-25T14:59:46.322884+0000","last_became_peered":"2024-04-25T14:59:46.322884+0000","last_unstale":"2024-04-25T14:59:47.291682+0000","last_undegraded":"2024-04-25T14:59:47.291682+0000","last_fullsized":"2024-04-25T14:59:47.291682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:51.192548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289234+0000","last_change":"2024-04-25T14:59:45.325553+0000","last_active":"2024-04-25T14:59:47.289234+0000","last_peered":"2024-04-25T14:59:47.289234+0000","last_clean":"2024-04-25T14:59:47.289234+0000","last_became_active":"2024-04-25T14:59:45.325201+0000","last_became_peered":"2024-04-25T14:59:45.325201+0000","last_unstale":"2024-04-25T14:59:47.289234+0000","last_undegraded":"2024-04-25T14:59:47.289234+0000","last_fullsized":"2024-04-25T14:59:47.289234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:05.068269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529098+0000","last_change":"2024-04-25T14:59:46.318252+0000","last_active":"2024-04-25T14:59:47.529098+0000","last_peered":"2024-04-25T14:59:47.529098+0000","last_clean":"2024-04-25T14:59:47.529098+0000","last_became_active":"2024-04-25T14:59:46.317601+0000","last_became_peered":"2024-04-25T14:59:46.317601+0000","last_unstale":"2024-04-25T14:59:47.529098+0000","last_undegraded":"2024-04-25T14:59:47.529098+0000","last_fullsized":"2024-04-25T14:59:47.529098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:33.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591610+0000","last_change":"2024-04-25T14:59:45.329430+0000","last_active":"2024-04-25T14:59:47.591610+0000","last_peered":"2024-04-25T14:59:47.591610+0000","last_clean":"2024-04-25T14:59:47.591610+0000","last_became_active":"2024-04-25T14:59:45.329140+0000","last_became_peered":"2024-04-25T14:59:45.329140+0000","last_unstale":"2024-04-25T14:59:47.591610+0000","last_undegraded":"2024-04-25T14:59:47.591610+0000","last_fullsized":"2024-04-25T14:59:47.591610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:49.114820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.580772+0000","last_change":"2024-04-25T14:59:46.299821+0000","last_active":"2024-04-25T14:59:47.580772+0000","last_peered":"2024-04-25T14:59:47.580772+0000","last_clean":"2024-04-25T14:59:47.580772+0000","last_became_active":"2024-04-25T14:59:46.299511+0000","last_became_peered":"2024-04-25T14:59:46.299511+0000","last_unstale":"2024-04-25T14:59:47.580772+0000","last_undegraded":"2024-04-25T14:59:47.580772+0000","last_fullsized":"2024-04-25T14:59:47.580772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:23.409333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.290852+0000","last_change":"2024-04-25T14:59:45.331464+0000","last_active":"2024-04-25T14:59:47.290852+0000","last_peered":"2024-04-25T14:59:47.290852+0000","last_clean":"2024-04-25T14:59:47.290852+0000","last_became_active":"2024-04-25T14:59:45.327879+0000","last_became_peered":"2024-04-25T14:59:45.327879+0000","last_unstale":"2024-04-25T14:59:47.290852+0000","last_undegraded":"2024-04-25T14:59:47.290852+0000","last_fullsized":"2024-04-25T14:59:47.290852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:50.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208125+0000","last_change":"2024-04-25T14:59:46.310621+0000","last_active":"2024-04-25T14:59:48.208125+0000","last_peered":"2024-04-25T14:59:48.208125+0000","last_clean":"2024-04-25T14:59:48.208125+0000","last_became_active":"2024-04-25T14:59:46.310292+0000","last_became_peered":"2024-04-25T14:59:46.310292+0000","last_unstale":"2024-04-25T14:59:48.208125+0000","last_undegraded":"2024-04-25T14:59:48.208125+0000","last_fullsized":"2024-04-25T14:59:48.208125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:20.035121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591550+0000","last_change":"2024-04-25T14:59:45.320737+0000","last_active":"2024-04-25T14:59:47.591550+0000","last_peered":"2024-04-25T14:59:47.591550+0000","last_clean":"2024-04-25T14:59:47.591550+0000","last_became_active":"2024-04-25T14:59:45.320404+0000","last_became_peered":"2024-04-25T14:59:45.320404+0000","last_unstale":"2024-04-25T14:59:47.591550+0000","last_undegraded":"2024-04-25T14:59:47.591550+0000","last_fullsized":"2024-04-25T14:59:47.591550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:04.016657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291684+0000","last_change":"2024-04-25T14:59:46.326568+0000","last_active":"2024-04-25T14:59:47.291684+0000","last_peered":"2024-04-25T14:59:47.291684+0000","last_clean":"2024-04-25T14:59:47.291684+0000","last_became_active":"2024-04-25T14:59:46.324019+0000","last_became_peered":"2024-04-25T14:59:46.324019+0000","last_unstale":"2024-04-25T14:59:47.291684+0000","last_undegraded":"2024-04-25T14:59:47.291684+0000","last_fullsized":"2024-04-25T14:59:47.291684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.985449+0000","last_change":"2024-04-25T14:59:45.316935+0000","last_active":"2024-04-25T14:59:47.985449+0000","last_peered":"2024-04-25T14:59:47.985449+0000","last_clean":"2024-04-25T14:59:47.985449+0000","last_became_active":"2024-04-25T14:59:45.316604+0000","last_became_peered":"2024-04-25T14:59:45.316604+0000","last_unstale":"2024-04-25T14:59:47.985449+0000","last_undegraded":"2024-04-25T14:59:47.985449+0000","last_fullsized":"2024-04-25T14:59:47.985449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:45.699926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592778+0000","last_change":"2024-04-25T14:59:46.316195+0000","last_active":"2024-04-25T14:59:47.592778+0000","last_peered":"2024-04-25T14:59:47.592778+0000","last_clean":"2024-04-25T14:59:47.592778+0000","last_became_active":"2024-04-25T14:59:46.315809+0000","last_became_peered":"2024-04-25T14:59:46.315809+0000","last_unstale":"2024-04-25T14:59:47.592778+0000","last_undegraded":"2024-04-25T14:59:47.592778+0000","last_fullsized":"2024-04-25T14:59:47.592778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:34.440058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591936+0000","last_change":"2024-04-25T14:59:45.325137+0000","last_active":"2024-04-25T14:59:47.591936+0000","last_peered":"2024-04-25T14:59:47.591936+0000","last_clean":"2024-04-25T14:59:47.591936+0000","last_became_active":"2024-04-25T14:59:45.324769+0000","last_became_peered":"2024-04-25T14:59:45.324769+0000","last_unstale":"2024-04-25T14:59:47.591936+0000","last_undegraded":"2024-04-25T14:59:47.591936+0000","last_fullsized":"2024-04-25T14:59:47.591936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:02.778853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593255+0000","last_change":"2024-04-25T14:59:46.304099+0000","last_active":"2024-04-25T14:59:47.593255+0000","last_peered":"2024-04-25T14:59:47.593255+0000","last_clean":"2024-04-25T14:59:47.593255+0000","last_became_active":"2024-04-25T14:59:46.303774+0000","last_became_peered":"2024-04-25T14:59:46.303774+0000","last_unstale":"2024-04-25T14:59:47.593255+0000","last_undegraded":"2024-04-25T14:59:47.593255+0000","last_fullsized":"2024-04-25T14:59:47.593255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.308212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291545+0000","last_change":"2024-04-25T14:59:45.322324+0000","last_active":"2024-04-25T14:59:47.291545+0000","last_peered":"2024-04-25T14:59:47.291545+0000","last_clean":"2024-04-25T14:59:47.291545+0000","last_became_active":"2024-04-25T14:59:45.322031+0000","last_became_peered":"2024-04-25T14:59:45.322031+0000","last_unstale":"2024-04-25T14:59:47.291545+0000","last_undegraded":"2024-04-25T14:59:47.291545+0000","last_fullsized":"2024-04-25T14:59:47.291545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.424928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592648+0000","last_change":"2024-04-25T14:59:46.315013+0000","last_active":"2024-04-25T14:59:47.592648+0000","last_peered":"2024-04-25T14:59:47.592648+0000","last_clean":"2024-04-25T14:59:47.592648+0000","last_became_active":"2024-04-25T14:59:46.314654+0000","last_became_peered":"2024-04-25T14:59:46.314654+0000","last_unstale":"2024-04-25T14:59:47.592648+0000","last_undegraded":"2024-04-25T14:59:47.592648+0000","last_fullsized":"2024-04-25T14:59:47.592648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:43.999674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594681+0000","last_change":"2024-04-25T14:59:45.335502+0000","last_active":"2024-04-25T14:59:47.594681+0000","last_peered":"2024-04-25T14:59:47.594681+0000","last_clean":"2024-04-25T14:59:47.594681+0000","last_became_active":"2024-04-25T14:59:45.335018+0000","last_became_peered":"2024-04-25T14:59:45.335018+0000","last_unstale":"2024-04-25T14:59:47.594681+0000","last_undegraded":"2024-04-25T14:59:47.594681+0000","last_fullsized":"2024-04-25T14:59:47.594681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:11.918001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593390+0000","last_change":"2024-04-25T14:59:46.304766+0000","last_active":"2024-04-25T14:59:47.593390+0000","last_peered":"2024-04-25T14:59:47.593390+0000","last_clean":"2024-04-25T14:59:47.593390+0000","last_became_active":"2024-04-25T14:59:46.304413+0000","last_became_peered":"2024-04-25T14:59:46.304413+0000","last_unstale":"2024-04-25T14:59:47.593390+0000","last_undegraded":"2024-04-25T14:59:47.593390+0000","last_fullsized":"2024-04-25T14:59:47.593390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208504+0000","last_change":"2024-04-25T14:59:46.305285+0000","last_active":"2024-04-25T14:59:48.208504+0000","last_peered":"2024-04-25T14:59:48.208504+0000","last_clean":"2024-04-25T14:59:48.208504+0000","last_became_active":"2024-04-25T14:59:46.304934+0000","last_became_peered":"2024-04-25T14:59:46.304934+0000","last_unstale":"2024-04-25T14:59:48.208504+0000","last_undegraded":"2024-04-25T14:59:48.208504+0000","last_fullsized":"2024-04-25T14:59:48.208504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:44.892921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591714+0000","last_change":"2024-04-25T14:59:46.303263+0000","last_active":"2024-04-25T14:59:47.591714+0000","last_peered":"2024-04-25T14:59:47.591714+0000","last_clean":"2024-04-25T14:59:47.591714+0000","last_became_active":"2024-04-25T14:59:46.302677+0000","last_became_peered":"2024-04-25T14:59:46.302677+0000","last_unstale":"2024-04-25T14:59:47.591714+0000","last_undegraded":"2024-04-25T14:59:47.591714+0000","last_fullsized":"2024-04-25T14:59:47.591714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:12.792516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591629+0000","last_change":"2024-04-25T14:59:46.319694+0000","last_active":"2024-04-25T14:59:47.591629+0000","last_peered":"2024-04-25T14:59:47.591629+0000","last_clean":"2024-04-25T14:59:47.591629+0000","last_became_active":"2024-04-25T14:59:46.319417+0000","last_became_peered":"2024-04-25T14:59:46.319417+0000","last_unstale":"2024-04-25T14:59:47.591629+0000","last_undegraded":"2024-04-25T14:59:47.591629+0000","last_fullsized":"2024-04-25T14:59:47.591629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.177498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.528907+0000","last_change":"2024-04-25T14:59:46.325705+0000","last_active":"2024-04-25T14:59:47.528907+0000","last_peered":"2024-04-25T14:59:47.528907+0000","last_clean":"2024-04-25T14:59:47.528907+0000","last_became_active":"2024-04-25T14:59:46.325386+0000","last_became_peered":"2024-04-25T14:59:46.325386+0000","last_unstale":"2024-04-25T14:59:47.528907+0000","last_undegraded":"2024-04-25T14:59:47.528907+0000","last_fullsized":"2024-04-25T14:59:47.528907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594782+0000","last_change":"2024-04-25T14:59:46.319167+0000","last_active":"2024-04-25T14:59:47.594782+0000","last_peered":"2024-04-25T14:59:47.594782+0000","last_clean":"2024-04-25T14:59:47.594782+0000","last_became_active":"2024-04-25T14:59:46.318804+0000","last_became_peered":"2024-04-25T14:59:46.318804+0000","last_unstale":"2024-04-25T14:59:47.594782+0000","last_undegraded":"2024-04-25T14:59:47.594782+0000","last_fullsized":"2024-04-25T14:59:47.594782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:34.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208294+0000","last_change":"2024-04-25T14:59:46.304554+0000","last_active":"2024-04-25T14:59:48.208294+0000","last_peered":"2024-04-25T14:59:48.208294+0000","last_clean":"2024-04-25T14:59:48.208294+0000","last_became_active":"2024-04-25T14:59:46.304062+0000","last_became_peered":"2024-04-25T14:59:46.304062+0000","last_unstale":"2024-04-25T14:59:48.208294+0000","last_undegraded":"2024-04-25T14:59:48.208294+0000","last_fullsized":"2024-04-25T14:59:48.208294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592222+0000","last_change":"2024-04-25T14:59:46.299104+0000","last_active":"2024-04-25T14:59:47.592222+0000","last_peered":"2024-04-25T14:59:47.592222+0000","last_clean":"2024-04-25T14:59:47.592222+0000","last_became_active":"2024-04-25T14:59:46.298765+0000","last_became_peered":"2024-04-25T14:59:46.298765+0000","last_unstale":"2024-04-25T14:59:47.592222+0000","last_undegraded":"2024-04-25T14:59:47.592222+0000","last_fullsized":"2024-04-25T14:59:47.592222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.494298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291455+0000","last_change":"2024-04-25T14:59:46.328885+0000","last_active":"2024-04-25T14:59:47.291455+0000","last_peered":"2024-04-25T14:59:47.291455+0000","last_clean":"2024-04-25T14:59:47.291455+0000","last_became_active":"2024-04-25T14:59:46.328375+0000","last_became_peered":"2024-04-25T14:59:46.328375+0000","last_unstale":"2024-04-25T14:59:47.291455+0000","last_undegraded":"2024-04-25T14:59:47.291455+0000","last_fullsized":"2024-04-25T14:59:47.291455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:08.959045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291358+0000","last_change":"2024-04-25T14:59:46.317403+0000","last_active":"2024-04-25T14:59:47.291358+0000","last_peered":"2024-04-25T14:59:47.291358+0000","last_clean":"2024-04-25T14:59:47.291358+0000","last_became_active":"2024-04-25T14:59:46.317100+0000","last_became_peered":"2024-04-25T14:59:46.317100+0000","last_unstale":"2024-04-25T14:59:47.291358+0000","last_undegraded":"2024-04-25T14:59:47.291358+0000","last_fullsized":"2024-04-25T14:59:47.291358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:23.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291766+0000","last_change":"2024-04-25T14:59:46.327147+0000","last_active":"2024-04-25T14:59:47.291766+0000","last_peered":"2024-04-25T14:59:47.291766+0000","last_clean":"2024-04-25T14:59:47.291766+0000","last_became_active":"2024-04-25T14:59:46.321857+0000","last_became_peered":"2024-04-25T14:59:46.321857+0000","last_unstale":"2024-04-25T14:59:47.291766+0000","last_undegraded":"2024-04-25T14:59:47.291766+0000","last_fullsized":"2024-04-25T14:59:47.291766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:46.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592139+0000","last_change":"2024-04-25T14:59:46.310841+0000","last_active":"2024-04-25T14:59:47.592139+0000","last_peered":"2024-04-25T14:59:47.592139+0000","last_clean":"2024-04-25T14:59:47.592139+0000","last_became_active":"2024-04-25T14:59:46.310446+0000","last_became_peered":"2024-04-25T14:59:46.310446+0000","last_unstale":"2024-04-25T14:59:47.592139+0000","last_undegraded":"2024-04-25T14:59:47.592139+0000","last_fullsized":"2024-04-25T14:59:47.592139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.594784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594789+0000","last_change":"2024-04-25T14:59:46.320715+0000","last_active":"2024-04-25T14:59:47.594789+0000","last_peered":"2024-04-25T14:59:47.594789+0000","last_clean":"2024-04-25T14:59:47.594789+0000","last_became_active":"2024-04-25T14:59:46.320238+0000","last_became_peered":"2024-04-25T14:59:46.320238+0000","last_unstale":"2024-04-25T14:59:47.594789+0000","last_undegraded":"2024-04-25T14:59:47.594789+0000","last_fullsized":"2024-04-25T14:59:47.594789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:55.387814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208192+0000","last_change":"2024-04-25T14:59:46.303772+0000","last_active":"2024-04-25T14:59:48.208192+0000","last_peered":"2024-04-25T14:59:48.208192+0000","last_clean":"2024-04-25T14:59:48.208192+0000","last_became_active":"2024-04-25T14:59:46.303435+0000","last_became_peered":"2024-04-25T14:59:46.303435+0000","last_unstale":"2024-04-25T14:59:48.208192+0000","last_undegraded":"2024-04-25T14:59:48.208192+0000","last_fullsized":"2024-04-25T14:59:48.208192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:06.692979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9010,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":48,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":20430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1540096,"data_stored":1033746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1519616,"data_stored":1032522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":449784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":937984,"data_stored":443218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149753,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1032122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":8406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:59:54.450 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 -- 172.21.15.94:0/143268421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5568105b80 msgr2=0x7f556810c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 --2- 172.21.15.94:0/143268421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5568105b80 0x7f556810c410 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5560009ca0 tx=0x7f556002a8b0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 -- 172.21.15.94:0/143268421 shutdown_connections 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 --2- 172.21.15.94:0/143268421 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5568105b80 0x7f556810c410 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 --2- 172.21.15.94:0/143268421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55681051c0 0x7f5568105640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 --2- 172.21.15.94:0/143268421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568103fd0 0x7f55681043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 -- 172.21.15.94:0/143268421 >> 172.21.15.94:0/143268421 conn(0x7f55680ff560 msgr2=0x7f5568101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 -- 172.21.15.94:0/143268421 shutdown_connections 2024-04-25T14:59:54.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.694+0000 7f5567e00640 1 -- 172.21.15.94:0/143268421 wait complete. 2024-04-25T14:59:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.695+0000 7f5567e00640 1 Processor -- start 2024-04-25T14:59:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.695+0000 7f5567e00640 1 -- start start 2024-04-25T14:59:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5568103fd0 0x7f556819a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55681051c0 0x7f556819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568105b80 0x7f55681a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f556810fb60 con 0x7f5568105b80 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f556810f9e0 con 0x7f55681051c0 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f556810fce0 con 0x7f5568103fd0 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568105b80 0x7f55681a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568105b80 0x7f55681a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35178/0 (socket says 172.21.15.94:35178) 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 -- 172.21.15.94:0/3408506410 learned_addr learned my addr 172.21.15.94:0/3408506410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f555e000640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55681051c0 0x7f556819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5566a00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5568103fd0 0x7f556819a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 -- 172.21.15.94:0/3408506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5568103fd0 msgr2=0x7f556819a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5568103fd0 0x7f556819a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 -- 172.21.15.94:0/3408506410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55681051c0 msgr2=0x7f556819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55681051c0 0x7f556819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.696+0000 7f5567400640 1 -- 172.21.15.94:0/3408506410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55681a1d90 con 0x7f5568105b80 2024-04-25T14:59:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.697+0000 7f5567400640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568105b80 0x7f55681a1720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5560003bd0 tx=0x7f5560003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.697+0000 7f555fe00640 1 -- 172.21.15.94:0/3408506410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560004450 con 0x7f5568105b80 2024-04-25T14:59:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.697+0000 7f555fe00640 1 -- 172.21.15.94:0/3408506410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5560040070 con 0x7f5568105b80 2024-04-25T14:59:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.697+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55681a2020 con 0x7f5568105b80 2024-04-25T14:59:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.697+0000 7f555fe00640 1 -- 172.21.15.94:0/3408506410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560031b30 con 0x7f5568105b80 2024-04-25T14:59:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.697+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a2570 con 0x7f5568105b80 2024-04-25T14:59:54.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.698+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f556810b370 con 0x7f5568105b80 2024-04-25T14:59:54.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.700+0000 7f555fe00640 1 -- 172.21.15.94:0/3408506410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5560031cd0 con 0x7f5568105b80 2024-04-25T14:59:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.702+0000 7f555fe00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f554c0a3c40 0x7f554c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.702+0000 7f555fe00640 1 -- 172.21.15.94:0/3408506410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f55600f14e0 con 0x7f5568105b80 2024-04-25T14:59:54.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.702+0000 7f5566a00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f554c0a3c40 0x7f554c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:54.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.703+0000 7f5566a00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f554c0a3c40 0x7f554c0a6100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5568105020 tx=0x7f5558005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:54.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.706+0000 7f555fe00640 1 -- 172.21.15.94:0/3408506410 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55600be020 con 0x7f5568105b80 2024-04-25T14:59:54.711 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:54 smithi094 ceph-mon[27862]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:54.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:54 smithi094 ceph-mon[27862]: pgmap v312: 97 pgs: 97 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T14:59:54.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:54 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2388703996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:54.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.907+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5568062390 con 0x7f554c0a3c40 2024-04-25T14:59:54.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.916+0000 7f555fe00640 1 -- 172.21.15.94:0/3408506410 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258655 (secure 0 0 0) 0x7f5568062390 con 0x7f554c0a3c40 2024-04-25T14:59:54.920 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T14:59:54.921 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T14:59:54.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f554c0a3c40 msgr2=0x7f554c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f554c0a3c40 0x7f554c0a6100 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5568105020 tx=0x7f5558005f90 comp rx=0 tx=0).stop 2024-04-25T14:59:54.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568105b80 msgr2=0x7f55681a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:54.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568105b80 0x7f55681a1720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5560003bd0 tx=0x7f5560003c00 comp rx=0 tx=0).stop 2024-04-25T14:59:54.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 shutdown_connections 2024-04-25T14:59:54.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f554c0a3c40 0x7f554c0a6100 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5568105b80 0x7f55681a1720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f55681051c0 0x7f556819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 --2- 172.21.15.94:0/3408506410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5568103fd0 0x7f556819a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:54.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.922+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 >> 172.21.15.94:0/3408506410 conn(0x7f55680ff560 msgr2=0x7f5568108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:54.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.923+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 shutdown_connections 2024-04-25T14:59:54.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:54.923+0000 7f5567e00640 1 -- 172.21.15.94:0/3408506410 wait complete. 2024-04-25T14:59:54.943 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-04-25T14:59:53.475335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599378,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":147,"num_read_kb":128,"num_write":229,"num_write_kb":2960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":259,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378500,"kb_used_data":12640,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124563324,"statfs":{"total":1151940427776,"available":1151552843776,"internally_reserved":0,"allocated":12943360,"data_stored":7083798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9010,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":48,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003103"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208436+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T14:59:48.208436+0000","last_peered":"2024-04-25T14:59:48.208436+0000","last_clean":"2024-04-25T14:59:48.208436+0000","last_became_active":"2024-04-25T14:59:46.311856+0000","last_became_peered":"2024-04-25T14:59:46.311856+0000","last_unstale":"2024-04-25T14:59:48.208436+0000","last_undegraded":"2024-04-25T14:59:48.208436+0000","last_fullsized":"2024-04-25T14:59:48.208436+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:07.469645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529013+0000","last_change":"2024-04-25T14:59:46.310292+0000","last_active":"2024-04-25T14:59:47.529013+0000","last_peered":"2024-04-25T14:59:47.529013+0000","last_clean":"2024-04-25T14:59:47.529013+0000","last_became_active":"2024-04-25T14:59:46.309812+0000","last_became_peered":"2024-04-25T14:59:46.309812+0000","last_unstale":"2024-04-25T14:59:47.529013+0000","last_undegraded":"2024-04-25T14:59:47.529013+0000","last_fullsized":"2024-04-25T14:59:47.529013+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:52.448406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529151+0000","last_change":"2024-04-25T14:59:46.300987+0000","last_active":"2024-04-25T14:59:47.529151+0000","last_peered":"2024-04-25T14:59:47.529151+0000","last_clean":"2024-04-25T14:59:47.529151+0000","last_became_active":"2024-04-25T14:59:46.299866+0000","last_became_peered":"2024-04-25T14:59:46.299866+0000","last_unstale":"2024-04-25T14:59:47.529151+0000","last_undegraded":"2024-04-25T14:59:47.529151+0000","last_fullsized":"2024-04-25T14:59:47.529151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:09.831131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289276+0000","last_change":"2024-04-25T14:59:46.304278+0000","last_active":"2024-04-25T14:59:47.289276+0000","last_peered":"2024-04-25T14:59:47.289276+0000","last_clean":"2024-04-25T14:59:47.289276+0000","last_became_active":"2024-04-25T14:59:46.303956+0000","last_became_peered":"2024-04-25T14:59:46.303956+0000","last_unstale":"2024-04-25T14:59:47.289276+0000","last_undegraded":"2024-04-25T14:59:47.289276+0000","last_fullsized":"2024-04-25T14:59:47.289276+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.752548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.580955+0000","last_change":"2024-04-25T14:59:46.323730+0000","last_active":"2024-04-25T14:59:47.580955+0000","last_peered":"2024-04-25T14:59:47.580955+0000","last_clean":"2024-04-25T14:59:47.580955+0000","last_became_active":"2024-04-25T14:59:46.323383+0000","last_became_peered":"2024-04-25T14:59:46.323383+0000","last_unstale":"2024-04-25T14:59:47.580955+0000","last_undegraded":"2024-04-25T14:59:47.580955+0000","last_fullsized":"2024-04-25T14:59:47.580955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.846392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592911+0000","last_change":"2024-04-25T14:59:46.317623+0000","last_active":"2024-04-25T14:59:47.592911+0000","last_peered":"2024-04-25T14:59:47.592911+0000","last_clean":"2024-04-25T14:59:47.592911+0000","last_became_active":"2024-04-25T14:59:46.317255+0000","last_became_peered":"2024-04-25T14:59:46.317255+0000","last_unstale":"2024-04-25T14:59:47.592911+0000","last_undegraded":"2024-04-25T14:59:47.592911+0000","last_fullsized":"2024-04-25T14:59:47.592911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:24.449615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592442+0000","last_change":"2024-04-25T14:59:46.318553+0000","last_active":"2024-04-25T14:59:47.592442+0000","last_peered":"2024-04-25T14:59:47.592442+0000","last_clean":"2024-04-25T14:59:47.592442+0000","last_became_active":"2024-04-25T14:59:46.318070+0000","last_became_peered":"2024-04-25T14:59:46.318070+0000","last_unstale":"2024-04-25T14:59:47.592442+0000","last_undegraded":"2024-04-25T14:59:47.592442+0000","last_fullsized":"2024-04-25T14:59:47.592442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:41.906324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291126+0000","last_change":"2024-04-25T14:59:46.327687+0000","last_active":"2024-04-25T14:59:47.291126+0000","last_peered":"2024-04-25T14:59:47.291126+0000","last_clean":"2024-04-25T14:59:47.291126+0000","last_became_active":"2024-04-25T14:59:46.325905+0000","last_became_peered":"2024-04-25T14:59:46.325905+0000","last_unstale":"2024-04-25T14:59:47.291126+0000","last_undegraded":"2024-04-25T14:59:47.291126+0000","last_fullsized":"2024-04-25T14:59:47.291126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:50.058801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592853+0000","last_change":"2024-04-25T14:59:46.304722+0000","last_active":"2024-04-25T14:59:47.592853+0000","last_peered":"2024-04-25T14:59:47.592853+0000","last_clean":"2024-04-25T14:59:47.592853+0000","last_became_active":"2024-04-25T14:59:46.304391+0000","last_became_peered":"2024-04-25T14:59:46.304391+0000","last_unstale":"2024-04-25T14:59:47.592853+0000","last_undegraded":"2024-04-25T14:59:47.592853+0000","last_fullsized":"2024-04-25T14:59:47.592853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:15.735670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593615+0000","last_change":"2024-04-25T14:59:46.307796+0000","last_active":"2024-04-25T14:59:47.593615+0000","last_peered":"2024-04-25T14:59:47.593615+0000","last_clean":"2024-04-25T14:59:47.593615+0000","last_became_active":"2024-04-25T14:59:46.306741+0000","last_became_peered":"2024-04-25T14:59:46.306741+0000","last_unstale":"2024-04-25T14:59:47.593615+0000","last_undegraded":"2024-04-25T14:59:47.593615+0000","last_fullsized":"2024-04-25T14:59:47.593615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:34.407272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593401+0000","last_change":"2024-04-25T14:59:46.306390+0000","last_active":"2024-04-25T14:59:47.593401+0000","last_peered":"2024-04-25T14:59:47.593401+0000","last_clean":"2024-04-25T14:59:47.593401+0000","last_became_active":"2024-04-25T14:59:46.306026+0000","last_became_peered":"2024-04-25T14:59:46.306026+0000","last_unstale":"2024-04-25T14:59:47.593401+0000","last_undegraded":"2024-04-25T14:59:47.593401+0000","last_fullsized":"2024-04-25T14:59:47.593401+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:59.340382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289247+0000","last_change":"2024-04-25T14:59:46.305840+0000","last_active":"2024-04-25T14:59:47.289247+0000","last_peered":"2024-04-25T14:59:47.289247+0000","last_clean":"2024-04-25T14:59:47.289247+0000","last_became_active":"2024-04-25T14:59:46.304867+0000","last_became_peered":"2024-04-25T14:59:46.304867+0000","last_unstale":"2024-04-25T14:59:47.289247+0000","last_undegraded":"2024-04-25T14:59:47.289247+0000","last_fullsized":"2024-04-25T14:59:47.289247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:58.569011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594966+0000","last_change":"2024-04-25T14:59:46.318044+0000","last_active":"2024-04-25T14:59:47.594966+0000","last_peered":"2024-04-25T14:59:47.594966+0000","last_clean":"2024-04-25T14:59:47.594966+0000","last_became_active":"2024-04-25T14:59:46.315935+0000","last_became_peered":"2024-04-25T14:59:46.315935+0000","last_unstale":"2024-04-25T14:59:47.594966+0000","last_undegraded":"2024-04-25T14:59:47.594966+0000","last_fullsized":"2024-04-25T14:59:47.594966+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:42.366802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.528670+0000","last_change":"2024-04-25T14:59:46.308565+0000","last_active":"2024-04-25T14:59:47.528670+0000","last_peered":"2024-04-25T14:59:47.528670+0000","last_clean":"2024-04-25T14:59:47.528670+0000","last_became_active":"2024-04-25T14:59:46.306198+0000","last_became_peered":"2024-04-25T14:59:46.306198+0000","last_unstale":"2024-04-25T14:59:47.528670+0000","last_undegraded":"2024-04-25T14:59:47.528670+0000","last_fullsized":"2024-04-25T14:59:47.528670+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:12.686162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.528490+0000","last_change":"2024-04-25T14:59:46.308313+0000","last_active":"2024-04-25T14:59:47.528490+0000","last_peered":"2024-04-25T14:59:47.528490+0000","last_clean":"2024-04-25T14:59:47.528490+0000","last_became_active":"2024-04-25T14:59:46.305974+0000","last_became_peered":"2024-04-25T14:59:46.305974+0000","last_unstale":"2024-04-25T14:59:47.528490+0000","last_undegraded":"2024-04-25T14:59:47.528490+0000","last_fullsized":"2024-04-25T14:59:47.528490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.841983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591641+0000","last_change":"2024-04-25T14:59:46.302726+0000","last_active":"2024-04-25T14:59:47.591641+0000","last_peered":"2024-04-25T14:59:47.591641+0000","last_clean":"2024-04-25T14:59:47.591641+0000","last_became_active":"2024-04-25T14:59:46.302406+0000","last_became_peered":"2024-04-25T14:59:46.302406+0000","last_unstale":"2024-04-25T14:59:47.591641+0000","last_undegraded":"2024-04-25T14:59:47.591641+0000","last_fullsized":"2024-04-25T14:59:47.591641+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208554+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T14:59:48.208554+0000","last_peered":"2024-04-25T14:59:48.208554+0000","last_clean":"2024-04-25T14:59:48.208554+0000","last_became_active":"2024-04-25T14:59:46.311822+0000","last_became_peered":"2024-04-25T14:59:46.311822+0000","last_unstale":"2024-04-25T14:59:48.208554+0000","last_undegraded":"2024-04-25T14:59:48.208554+0000","last_fullsized":"2024-04-25T14:59:48.208554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.128537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578371+0000","last_change":"2024-04-25T14:59:46.313816+0000","last_active":"2024-04-25T14:59:47.578371+0000","last_peered":"2024-04-25T14:59:47.578371+0000","last_clean":"2024-04-25T14:59:47.578371+0000","last_became_active":"2024-04-25T14:59:46.313462+0000","last_became_peered":"2024-04-25T14:59:46.313462+0000","last_unstale":"2024-04-25T14:59:47.578371+0000","last_undegraded":"2024-04-25T14:59:47.578371+0000","last_fullsized":"2024-04-25T14:59:47.578371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:29.402113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592037+0000","last_change":"2024-04-25T14:59:46.317972+0000","last_active":"2024-04-25T14:59:47.592037+0000","last_peered":"2024-04-25T14:59:47.592037+0000","last_clean":"2024-04-25T14:59:47.592037+0000","last_became_active":"2024-04-25T14:59:46.317466+0000","last_became_peered":"2024-04-25T14:59:46.317466+0000","last_unstale":"2024-04-25T14:59:47.592037+0000","last_undegraded":"2024-04-25T14:59:47.592037+0000","last_fullsized":"2024-04-25T14:59:47.592037+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:03.706663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578572+0000","last_change":"2024-04-25T14:59:45.320752+0000","last_active":"2024-04-25T14:59:47.578572+0000","last_peered":"2024-04-25T14:59:47.578572+0000","last_clean":"2024-04-25T14:59:47.578572+0000","last_became_active":"2024-04-25T14:59:45.319463+0000","last_became_peered":"2024-04-25T14:59:45.319463+0000","last_unstale":"2024-04-25T14:59:47.578572+0000","last_undegraded":"2024-04-25T14:59:47.578572+0000","last_fullsized":"2024-04-25T14:59:47.578572+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:52.608899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.292922+0000","last_change":"2024-04-25T14:59:46.323802+0000","last_active":"2024-04-25T14:59:47.292922+0000","last_peered":"2024-04-25T14:59:47.292922+0000","last_clean":"2024-04-25T14:59:47.292922+0000","last_became_active":"2024-04-25T14:59:46.323498+0000","last_became_peered":"2024-04-25T14:59:46.323498+0000","last_unstale":"2024-04-25T14:59:47.292922+0000","last_undegraded":"2024-04-25T14:59:47.292922+0000","last_fullsized":"2024-04-25T14:59:47.292922+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:38.236936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.292883+0000","last_change":"2024-04-25T14:59:45.335156+0000","last_active":"2024-04-25T14:59:47.292883+0000","last_peered":"2024-04-25T14:59:47.292883+0000","last_clean":"2024-04-25T14:59:47.292883+0000","last_became_active":"2024-04-25T14:59:45.333589+0000","last_became_peered":"2024-04-25T14:59:45.333589+0000","last_unstale":"2024-04-25T14:59:47.292883+0000","last_undegraded":"2024-04-25T14:59:47.292883+0000","last_fullsized":"2024-04-25T14:59:47.292883+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529610+0000","last_change":"2024-04-25T14:59:46.319165+0000","last_active":"2024-04-25T14:59:47.529610+0000","last_peered":"2024-04-25T14:59:47.529610+0000","last_clean":"2024-04-25T14:59:47.529610+0000","last_became_active":"2024-04-25T14:59:46.318801+0000","last_became_peered":"2024-04-25T14:59:46.318801+0000","last_unstale":"2024-04-25T14:59:47.529610+0000","last_undegraded":"2024-04-25T14:59:47.529610+0000","last_fullsized":"2024-04-25T14:59:47.529610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:01.806099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581387+0000","last_change":"2024-04-25T14:59:45.317956+0000","last_active":"2024-04-25T14:59:47.581387+0000","last_peered":"2024-04-25T14:59:47.581387+0000","last_clean":"2024-04-25T14:59:47.581387+0000","last_became_active":"2024-04-25T14:59:45.317655+0000","last_became_peered":"2024-04-25T14:59:45.317655+0000","last_unstale":"2024-04-25T14:59:47.581387+0000","last_undegraded":"2024-04-25T14:59:47.581387+0000","last_fullsized":"2024-04-25T14:59:47.581387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:59.008411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592332+0000","last_change":"2024-04-25T14:59:46.311659+0000","last_active":"2024-04-25T14:59:47.592332+0000","last_peered":"2024-04-25T14:59:47.592332+0000","last_clean":"2024-04-25T14:59:47.592332+0000","last_became_active":"2024-04-25T14:59:46.311287+0000","last_became_peered":"2024-04-25T14:59:46.311287+0000","last_unstale":"2024-04-25T14:59:47.592332+0000","last_undegraded":"2024-04-25T14:59:47.592332+0000","last_fullsized":"2024-04-25T14:59:47.592332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:29.434223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581133+0000","last_change":"2024-04-25T14:59:45.317658+0000","last_active":"2024-04-25T14:59:47.581133+0000","last_peered":"2024-04-25T14:59:47.581133+0000","last_clean":"2024-04-25T14:59:47.581133+0000","last_became_active":"2024-04-25T14:59:45.317311+0000","last_became_peered":"2024-04-25T14:59:45.317311+0000","last_unstale":"2024-04-25T14:59:47.581133+0000","last_undegraded":"2024-04-25T14:59:47.581133+0000","last_fullsized":"2024-04-25T14:59:47.581133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.602412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291939+0000","last_change":"2024-04-25T14:59:46.305456+0000","last_active":"2024-04-25T14:59:47.291939+0000","last_peered":"2024-04-25T14:59:47.291939+0000","last_clean":"2024-04-25T14:59:47.291939+0000","last_became_active":"2024-04-25T14:59:46.305143+0000","last_became_peered":"2024-04-25T14:59:46.305143+0000","last_unstale":"2024-04-25T14:59:47.291939+0000","last_undegraded":"2024-04-25T14:59:47.291939+0000","last_fullsized":"2024-04-25T14:59:47.291939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:03.941200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291489+0000","last_change":"2024-04-25T14:59:45.315407+0000","last_active":"2024-04-25T14:59:47.291489+0000","last_peered":"2024-04-25T14:59:47.291489+0000","last_clean":"2024-04-25T14:59:47.291489+0000","last_became_active":"2024-04-25T14:59:45.315070+0000","last_became_peered":"2024-04-25T14:59:45.315070+0000","last_unstale":"2024-04-25T14:59:47.291489+0000","last_undegraded":"2024-04-25T14:59:47.291489+0000","last_fullsized":"2024-04-25T14:59:47.291489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:32.492027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581007+0000","last_change":"2024-04-25T14:59:46.318218+0000","last_active":"2024-04-25T14:59:47.581007+0000","last_peered":"2024-04-25T14:59:47.581007+0000","last_clean":"2024-04-25T14:59:47.581007+0000","last_became_active":"2024-04-25T14:59:46.317855+0000","last_became_peered":"2024-04-25T14:59:46.317855+0000","last_unstale":"2024-04-25T14:59:47.581007+0000","last_undegraded":"2024-04-25T14:59:47.581007+0000","last_fullsized":"2024-04-25T14:59:47.581007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:11.712691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592057+0000","last_change":"2024-04-25T14:59:45.319624+0000","last_active":"2024-04-25T14:59:47.592057+0000","last_peered":"2024-04-25T14:59:47.592057+0000","last_clean":"2024-04-25T14:59:47.592057+0000","last_became_active":"2024-04-25T14:59:45.319310+0000","last_became_peered":"2024-04-25T14:59:45.319310+0000","last_unstale":"2024-04-25T14:59:47.592057+0000","last_undegraded":"2024-04-25T14:59:47.592057+0000","last_fullsized":"2024-04-25T14:59:47.592057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:40.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291556+0000","last_change":"2024-04-25T14:59:46.317382+0000","last_active":"2024-04-25T14:59:47.291556+0000","last_peered":"2024-04-25T14:59:47.291556+0000","last_clean":"2024-04-25T14:59:47.291556+0000","last_became_active":"2024-04-25T14:59:46.316993+0000","last_became_peered":"2024-04-25T14:59:46.316993+0000","last_unstale":"2024-04-25T14:59:47.291556+0000","last_undegraded":"2024-04-25T14:59:47.291556+0000","last_fullsized":"2024-04-25T14:59:47.291556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:06.618964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593927+0000","last_change":"2024-04-25T14:59:45.322731+0000","last_active":"2024-04-25T14:59:47.593927+0000","last_peered":"2024-04-25T14:59:47.593927+0000","last_clean":"2024-04-25T14:59:47.593927+0000","last_became_active":"2024-04-25T14:59:45.322360+0000","last_became_peered":"2024-04-25T14:59:45.322360+0000","last_unstale":"2024-04-25T14:59:47.593927+0000","last_undegraded":"2024-04-25T14:59:47.593927+0000","last_fullsized":"2024-04-25T14:59:47.593927+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:02.890016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291785+0000","last_change":"2024-04-25T14:59:46.327195+0000","last_active":"2024-04-25T14:59:47.291785+0000","last_peered":"2024-04-25T14:59:47.291785+0000","last_clean":"2024-04-25T14:59:47.291785+0000","last_became_active":"2024-04-25T14:59:46.322053+0000","last_became_peered":"2024-04-25T14:59:46.322053+0000","last_unstale":"2024-04-25T14:59:47.291785+0000","last_undegraded":"2024-04-25T14:59:47.291785+0000","last_fullsized":"2024-04-25T14:59:47.291785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:35.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291231+0000","last_change":"2024-04-25T14:59:45.315688+0000","last_active":"2024-04-25T14:59:47.291231+0000","last_peered":"2024-04-25T14:59:47.291231+0000","last_clean":"2024-04-25T14:59:47.291231+0000","last_became_active":"2024-04-25T14:59:45.315074+0000","last_became_peered":"2024-04-25T14:59:45.315074+0000","last_unstale":"2024-04-25T14:59:47.291231+0000","last_undegraded":"2024-04-25T14:59:47.291231+0000","last_fullsized":"2024-04-25T14:59:47.291231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:49.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.577841+0000","last_change":"2024-04-25T14:59:46.322874+0000","last_active":"2024-04-25T14:59:47.577841+0000","last_peered":"2024-04-25T14:59:47.577841+0000","last_clean":"2024-04-25T14:59:47.577841+0000","last_became_active":"2024-04-25T14:59:46.322492+0000","last_became_peered":"2024-04-25T14:59:46.322492+0000","last_unstale":"2024-04-25T14:59:47.577841+0000","last_undegraded":"2024-04-25T14:59:47.577841+0000","last_fullsized":"2024-04-25T14:59:47.577841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:21.294792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593230+0000","last_change":"2024-04-25T14:59:45.322448+0000","last_active":"2024-04-25T14:59:47.593230+0000","last_peered":"2024-04-25T14:59:47.593230+0000","last_clean":"2024-04-25T14:59:47.593230+0000","last_became_active":"2024-04-25T14:59:45.322127+0000","last_became_peered":"2024-04-25T14:59:45.322127+0000","last_unstale":"2024-04-25T14:59:47.593230+0000","last_undegraded":"2024-04-25T14:59:47.593230+0000","last_fullsized":"2024-04-25T14:59:47.593230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.284668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592432+0000","last_change":"2024-04-25T14:59:46.318371+0000","last_active":"2024-04-25T14:59:47.592432+0000","last_peered":"2024-04-25T14:59:47.592432+0000","last_clean":"2024-04-25T14:59:47.592432+0000","last_became_active":"2024-04-25T14:59:46.317819+0000","last_became_peered":"2024-04-25T14:59:46.317819+0000","last_unstale":"2024-04-25T14:59:47.592432+0000","last_undegraded":"2024-04-25T14:59:47.592432+0000","last_fullsized":"2024-04-25T14:59:47.592432+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.047982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592846+0000","last_change":"2024-04-25T14:59:45.318286+0000","last_active":"2024-04-25T14:59:47.592846+0000","last_peered":"2024-04-25T14:59:47.592846+0000","last_clean":"2024-04-25T14:59:47.592846+0000","last_became_active":"2024-04-25T14:59:45.317787+0000","last_became_peered":"2024-04-25T14:59:45.317787+0000","last_unstale":"2024-04-25T14:59:47.592846+0000","last_undegraded":"2024-04-25T14:59:47.592846+0000","last_fullsized":"2024-04-25T14:59:47.592846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:06.508164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529494+0000","last_change":"2024-04-25T14:59:46.309331+0000","last_active":"2024-04-25T14:59:47.529494+0000","last_peered":"2024-04-25T14:59:47.529494+0000","last_clean":"2024-04-25T14:59:47.529494+0000","last_became_active":"2024-04-25T14:59:46.306252+0000","last_became_peered":"2024-04-25T14:59:46.306252+0000","last_unstale":"2024-04-25T14:59:47.529494+0000","last_undegraded":"2024-04-25T14:59:47.529494+0000","last_fullsized":"2024-04-25T14:59:47.529494+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:53.896138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291834+0000","last_change":"2024-04-25T14:59:45.327951+0000","last_active":"2024-04-25T14:59:47.291834+0000","last_peered":"2024-04-25T14:59:47.291834+0000","last_clean":"2024-04-25T14:59:47.291834+0000","last_became_active":"2024-04-25T14:59:45.327660+0000","last_became_peered":"2024-04-25T14:59:45.327660+0000","last_unstale":"2024-04-25T14:59:47.291834+0000","last_undegraded":"2024-04-25T14:59:47.291834+0000","last_fullsized":"2024-04-25T14:59:47.291834+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:29.862798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592221+0000","last_change":"2024-04-25T14:59:46.304038+0000","last_active":"2024-04-25T14:59:47.592221+0000","last_peered":"2024-04-25T14:59:47.592221+0000","last_clean":"2024-04-25T14:59:47.592221+0000","last_became_active":"2024-04-25T14:59:46.303713+0000","last_became_peered":"2024-04-25T14:59:46.303713+0000","last_unstale":"2024-04-25T14:59:47.592221+0000","last_undegraded":"2024-04-25T14:59:47.592221+0000","last_fullsized":"2024-04-25T14:59:47.592221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:30.476591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'9","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:51.126826+0000","last_change":"2024-04-25T14:59:45.315738+0000","last_active":"2024-04-25T14:59:51.126826+0000","last_peered":"2024-04-25T14:59:51.126826+0000","last_clean":"2024-04-25T14:59:51.126826+0000","last_became_active":"2024-04-25T14:59:45.315423+0000","last_became_peered":"2024-04-25T14:59:45.315423+0000","last_unstale":"2024-04-25T14:59:51.126826+0000","last_undegraded":"2024-04-25T14:59:51.126826+0000","last_fullsized":"2024-04-25T14:59:51.126826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:33.135348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8212,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581304+0000","last_change":"2024-04-25T14:59:46.308085+0000","last_active":"2024-04-25T14:59:47.581304+0000","last_peered":"2024-04-25T14:59:47.581304+0000","last_clean":"2024-04-25T14:59:47.581304+0000","last_became_active":"2024-04-25T14:59:46.307739+0000","last_became_peered":"2024-04-25T14:59:46.307739+0000","last_unstale":"2024-04-25T14:59:47.581304+0000","last_undegraded":"2024-04-25T14:59:47.581304+0000","last_fullsized":"2024-04-25T14:59:47.581304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:57.949147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591667+0000","last_change":"2024-04-25T14:59:45.327058+0000","last_active":"2024-04-25T14:59:47.591667+0000","last_peered":"2024-04-25T14:59:47.591667+0000","last_clean":"2024-04-25T14:59:47.591667+0000","last_became_active":"2024-04-25T14:59:45.326817+0000","last_became_peered":"2024-04-25T14:59:45.326817+0000","last_unstale":"2024-04-25T14:59:47.591667+0000","last_undegraded":"2024-04-25T14:59:47.591667+0000","last_fullsized":"2024-04-25T14:59:47.591667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:51.087966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594493+0000","last_change":"2024-04-25T14:59:46.328901+0000","last_active":"2024-04-25T14:59:47.594493+0000","last_peered":"2024-04-25T14:59:47.594493+0000","last_clean":"2024-04-25T14:59:47.594493+0000","last_became_active":"2024-04-25T14:59:46.328496+0000","last_became_peered":"2024-04-25T14:59:46.328496+0000","last_unstale":"2024-04-25T14:59:47.594493+0000","last_undegraded":"2024-04-25T14:59:47.594493+0000","last_fullsized":"2024-04-25T14:59:47.594493+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:48.149719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291330+0000","last_change":"2024-04-25T14:59:45.316673+0000","last_active":"2024-04-25T14:59:47.291330+0000","last_peered":"2024-04-25T14:59:47.291330+0000","last_clean":"2024-04-25T14:59:47.291330+0000","last_became_active":"2024-04-25T14:59:45.316280+0000","last_became_peered":"2024-04-25T14:59:45.316280+0000","last_unstale":"2024-04-25T14:59:47.291330+0000","last_undegraded":"2024-04-25T14:59:47.291330+0000","last_fullsized":"2024-04-25T14:59:47.291330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:51.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592813+0000","last_change":"2024-04-25T14:59:46.323821+0000","last_active":"2024-04-25T14:59:47.592813+0000","last_peered":"2024-04-25T14:59:47.592813+0000","last_clean":"2024-04-25T14:59:47.592813+0000","last_became_active":"2024-04-25T14:59:46.323434+0000","last_became_peered":"2024-04-25T14:59:46.323434+0000","last_unstale":"2024-04-25T14:59:47.592813+0000","last_undegraded":"2024-04-25T14:59:47.592813+0000","last_fullsized":"2024-04-25T14:59:47.592813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:20.983270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.288973+0000","last_change":"2024-04-25T14:59:45.317953+0000","last_active":"2024-04-25T14:59:47.288973+0000","last_peered":"2024-04-25T14:59:47.288973+0000","last_clean":"2024-04-25T14:59:47.288973+0000","last_became_active":"2024-04-25T14:59:45.317617+0000","last_became_peered":"2024-04-25T14:59:45.317617+0000","last_unstale":"2024-04-25T14:59:47.288973+0000","last_undegraded":"2024-04-25T14:59:47.288973+0000","last_fullsized":"2024-04-25T14:59:47.288973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:44.235492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591705+0000","last_change":"2024-04-25T14:59:46.305929+0000","last_active":"2024-04-25T14:59:47.591705+0000","last_peered":"2024-04-25T14:59:47.591705+0000","last_clean":"2024-04-25T14:59:47.591705+0000","last_became_active":"2024-04-25T14:59:46.305604+0000","last_became_peered":"2024-04-25T14:59:46.305604+0000","last_unstale":"2024-04-25T14:59:47.591705+0000","last_undegraded":"2024-04-25T14:59:47.591705+0000","last_fullsized":"2024-04-25T14:59:47.591705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:37.340009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.580741+0000","last_change":"2024-04-25T14:59:45.321385+0000","last_active":"2024-04-25T14:59:47.580741+0000","last_peered":"2024-04-25T14:59:47.580741+0000","last_clean":"2024-04-25T14:59:47.580741+0000","last_became_active":"2024-04-25T14:59:45.321057+0000","last_became_peered":"2024-04-25T14:59:45.321057+0000","last_unstale":"2024-04-25T14:59:47.580741+0000","last_undegraded":"2024-04-25T14:59:47.580741+0000","last_fullsized":"2024-04-25T14:59:47.580741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:24.907017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291274+0000","last_change":"2024-04-25T14:59:46.306259+0000","last_active":"2024-04-25T14:59:47.291274+0000","last_peered":"2024-04-25T14:59:47.291274+0000","last_clean":"2024-04-25T14:59:47.291274+0000","last_became_active":"2024-04-25T14:59:46.305938+0000","last_became_peered":"2024-04-25T14:59:46.305938+0000","last_unstale":"2024-04-25T14:59:47.291274+0000","last_undegraded":"2024-04-25T14:59:47.291274+0000","last_fullsized":"2024-04-25T14:59:47.291274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:24.931749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.985502+0000","last_change":"2024-04-25T14:59:45.318291+0000","last_active":"2024-04-25T14:59:47.985502+0000","last_peered":"2024-04-25T14:59:47.985502+0000","last_clean":"2024-04-25T14:59:47.985502+0000","last_became_active":"2024-04-25T14:59:45.317590+0000","last_became_peered":"2024-04-25T14:59:45.317590+0000","last_unstale":"2024-04-25T14:59:47.985502+0000","last_undegraded":"2024-04-25T14:59:47.985502+0000","last_fullsized":"2024-04-25T14:59:47.985502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208347+0000","last_change":"2024-04-25T14:59:46.303503+0000","last_active":"2024-04-25T14:59:48.208347+0000","last_peered":"2024-04-25T14:59:48.208347+0000","last_clean":"2024-04-25T14:59:48.208347+0000","last_became_active":"2024-04-25T14:59:46.303127+0000","last_became_peered":"2024-04-25T14:59:46.303127+0000","last_unstale":"2024-04-25T14:59:48.208347+0000","last_undegraded":"2024-04-25T14:59:48.208347+0000","last_fullsized":"2024-04-25T14:59:48.208347+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:46.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578157+0000","last_change":"2024-04-25T14:59:45.330010+0000","last_active":"2024-04-25T14:59:47.578157+0000","last_peered":"2024-04-25T14:59:47.578157+0000","last_clean":"2024-04-25T14:59:47.578157+0000","last_became_active":"2024-04-25T14:59:45.329225+0000","last_became_peered":"2024-04-25T14:59:45.329225+0000","last_unstale":"2024-04-25T14:59:47.578157+0000","last_undegraded":"2024-04-25T14:59:47.578157+0000","last_fullsized":"2024-04-25T14:59:47.578157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:01.623458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593114+0000","last_change":"2024-04-25T14:59:46.320003+0000","last_active":"2024-04-25T14:59:47.593114+0000","last_peered":"2024-04-25T14:59:47.593114+0000","last_clean":"2024-04-25T14:59:47.593114+0000","last_became_active":"2024-04-25T14:59:46.319600+0000","last_became_peered":"2024-04-25T14:59:46.319600+0000","last_unstale":"2024-04-25T14:59:47.593114+0000","last_undegraded":"2024-04-25T14:59:47.593114+0000","last_fullsized":"2024-04-25T14:59:47.593114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.916018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592595+0000","last_change":"2024-04-25T14:59:45.325233+0000","last_active":"2024-04-25T14:59:47.592595+0000","last_peered":"2024-04-25T14:59:47.592595+0000","last_clean":"2024-04-25T14:59:47.592595+0000","last_became_active":"2024-04-25T14:59:45.324826+0000","last_became_peered":"2024-04-25T14:59:45.324826+0000","last_unstale":"2024-04-25T14:59:47.592595+0000","last_undegraded":"2024-04-25T14:59:47.592595+0000","last_fullsized":"2024-04-25T14:59:47.592595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.422559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593303+0000","last_change":"2024-04-25T14:59:46.314552+0000","last_active":"2024-04-25T14:59:47.593303+0000","last_peered":"2024-04-25T14:59:47.593303+0000","last_clean":"2024-04-25T14:59:47.593303+0000","last_became_active":"2024-04-25T14:59:46.314159+0000","last_became_peered":"2024-04-25T14:59:46.314159+0000","last_unstale":"2024-04-25T14:59:47.593303+0000","last_undegraded":"2024-04-25T14:59:47.593303+0000","last_fullsized":"2024-04-25T14:59:47.593303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:06.731960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291682+0000","last_change":"2024-04-25T14:59:46.303354+0000","last_active":"2024-04-25T14:59:47.291682+0000","last_peered":"2024-04-25T14:59:47.291682+0000","last_clean":"2024-04-25T14:59:47.291682+0000","last_became_active":"2024-04-25T14:59:46.302869+0000","last_became_peered":"2024-04-25T14:59:46.302869+0000","last_unstale":"2024-04-25T14:59:47.291682+0000","last_undegraded":"2024-04-25T14:59:47.291682+0000","last_fullsized":"2024-04-25T14:59:47.291682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.796902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593646+0000","last_change":"2024-04-25T14:59:45.336686+0000","last_active":"2024-04-25T14:59:47.593646+0000","last_peered":"2024-04-25T14:59:47.593646+0000","last_clean":"2024-04-25T14:59:47.593646+0000","last_became_active":"2024-04-25T14:59:45.335828+0000","last_became_peered":"2024-04-25T14:59:45.335828+0000","last_unstale":"2024-04-25T14:59:47.593646+0000","last_undegraded":"2024-04-25T14:59:47.593646+0000","last_fullsized":"2024-04-25T14:59:47.593646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.676117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291807+0000","last_change":"2024-04-25T14:59:46.319541+0000","last_active":"2024-04-25T14:59:47.291807+0000","last_peered":"2024-04-25T14:59:47.291807+0000","last_clean":"2024-04-25T14:59:47.291807+0000","last_became_active":"2024-04-25T14:59:46.317922+0000","last_became_peered":"2024-04-25T14:59:46.317922+0000","last_unstale":"2024-04-25T14:59:47.291807+0000","last_undegraded":"2024-04-25T14:59:47.291807+0000","last_fullsized":"2024-04-25T14:59:47.291807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.454075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289000+0000","last_change":"2024-04-25T14:59:45.313817+0000","last_active":"2024-04-25T14:59:47.289000+0000","last_peered":"2024-04-25T14:59:47.289000+0000","last_clean":"2024-04-25T14:59:47.289000+0000","last_became_active":"2024-04-25T14:59:45.313507+0000","last_became_peered":"2024-04-25T14:59:45.313507+0000","last_unstale":"2024-04-25T14:59:47.289000+0000","last_undegraded":"2024-04-25T14:59:47.289000+0000","last_fullsized":"2024-04-25T14:59:47.289000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:18.126351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591814+0000","last_change":"2024-04-25T14:58:29.935018+0000","last_active":"2024-04-25T14:59:47.591814+0000","last_peered":"2024-04-25T14:59:47.591814+0000","last_clean":"2024-04-25T14:59:47.591814+0000","last_became_active":"2024-04-25T14:58:29.934454+0000","last_became_peered":"2024-04-25T14:58:29.934454+0000","last_unstale":"2024-04-25T14:59:47.591814+0000","last_undegraded":"2024-04-25T14:59:47.591814+0000","last_fullsized":"2024-04-25T14:59:47.591814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_clean_scrub_stamp":"2024-04-25T14:54:46.455802+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:09.308783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593901+0000","last_change":"2024-04-25T14:59:46.319680+0000","last_active":"2024-04-25T14:59:47.593901+0000","last_peered":"2024-04-25T14:59:47.593901+0000","last_clean":"2024-04-25T14:59:47.593901+0000","last_became_active":"2024-04-25T14:59:46.319225+0000","last_became_peered":"2024-04-25T14:59:46.319225+0000","last_unstale":"2024-04-25T14:59:47.593901+0000","last_undegraded":"2024-04-25T14:59:47.593901+0000","last_fullsized":"2024-04-25T14:59:47.593901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:34.629244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592073+0000","last_change":"2024-04-25T14:59:45.318087+0000","last_active":"2024-04-25T14:59:47.592073+0000","last_peered":"2024-04-25T14:59:47.592073+0000","last_clean":"2024-04-25T14:59:47.592073+0000","last_became_active":"2024-04-25T14:59:45.317745+0000","last_became_peered":"2024-04-25T14:59:45.317745+0000","last_unstale":"2024-04-25T14:59:47.592073+0000","last_undegraded":"2024-04-25T14:59:47.592073+0000","last_fullsized":"2024-04-25T14:59:47.592073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:17.202201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.581234+0000","last_change":"2024-04-25T14:59:45.332358+0000","last_active":"2024-04-25T14:59:47.581234+0000","last_peered":"2024-04-25T14:59:47.581234+0000","last_clean":"2024-04-25T14:59:47.581234+0000","last_became_active":"2024-04-25T14:59:45.332089+0000","last_became_peered":"2024-04-25T14:59:45.332089+0000","last_unstale":"2024-04-25T14:59:47.581234+0000","last_undegraded":"2024-04-25T14:59:47.581234+0000","last_fullsized":"2024-04-25T14:59:47.581234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:50.937548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594844+0000","last_change":"2024-04-25T14:59:46.320945+0000","last_active":"2024-04-25T14:59:47.594844+0000","last_peered":"2024-04-25T14:59:47.594844+0000","last_clean":"2024-04-25T14:59:47.594844+0000","last_became_active":"2024-04-25T14:59:46.320602+0000","last_became_peered":"2024-04-25T14:59:46.320602+0000","last_unstale":"2024-04-25T14:59:47.594844+0000","last_undegraded":"2024-04-25T14:59:47.594844+0000","last_fullsized":"2024-04-25T14:59:47.594844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:32.282647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.578436+0000","last_change":"2024-04-25T14:59:45.334325+0000","last_active":"2024-04-25T14:59:47.578436+0000","last_peered":"2024-04-25T14:59:47.578436+0000","last_clean":"2024-04-25T14:59:47.578436+0000","last_became_active":"2024-04-25T14:59:45.333969+0000","last_became_peered":"2024-04-25T14:59:45.333969+0000","last_unstale":"2024-04-25T14:59:47.578436+0000","last_undegraded":"2024-04-25T14:59:47.578436+0000","last_fullsized":"2024-04-25T14:59:47.578436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:02.694267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291682+0000","last_change":"2024-04-25T14:59:46.323686+0000","last_active":"2024-04-25T14:59:47.291682+0000","last_peered":"2024-04-25T14:59:47.291682+0000","last_clean":"2024-04-25T14:59:47.291682+0000","last_became_active":"2024-04-25T14:59:46.322884+0000","last_became_peered":"2024-04-25T14:59:46.322884+0000","last_unstale":"2024-04-25T14:59:47.291682+0000","last_undegraded":"2024-04-25T14:59:47.291682+0000","last_fullsized":"2024-04-25T14:59:47.291682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:51.192548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.289234+0000","last_change":"2024-04-25T14:59:45.325553+0000","last_active":"2024-04-25T14:59:47.289234+0000","last_peered":"2024-04-25T14:59:47.289234+0000","last_clean":"2024-04-25T14:59:47.289234+0000","last_became_active":"2024-04-25T14:59:45.325201+0000","last_became_peered":"2024-04-25T14:59:45.325201+0000","last_unstale":"2024-04-25T14:59:47.289234+0000","last_undegraded":"2024-04-25T14:59:47.289234+0000","last_fullsized":"2024-04-25T14:59:47.289234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:05.068269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.529098+0000","last_change":"2024-04-25T14:59:46.318252+0000","last_active":"2024-04-25T14:59:47.529098+0000","last_peered":"2024-04-25T14:59:47.529098+0000","last_clean":"2024-04-25T14:59:47.529098+0000","last_became_active":"2024-04-25T14:59:46.317601+0000","last_became_peered":"2024-04-25T14:59:46.317601+0000","last_unstale":"2024-04-25T14:59:47.529098+0000","last_undegraded":"2024-04-25T14:59:47.529098+0000","last_fullsized":"2024-04-25T14:59:47.529098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:33.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591610+0000","last_change":"2024-04-25T14:59:45.329430+0000","last_active":"2024-04-25T14:59:47.591610+0000","last_peered":"2024-04-25T14:59:47.591610+0000","last_clean":"2024-04-25T14:59:47.591610+0000","last_became_active":"2024-04-25T14:59:45.329140+0000","last_became_peered":"2024-04-25T14:59:45.329140+0000","last_unstale":"2024-04-25T14:59:47.591610+0000","last_undegraded":"2024-04-25T14:59:47.591610+0000","last_fullsized":"2024-04-25T14:59:47.591610+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:49.114820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.580772+0000","last_change":"2024-04-25T14:59:46.299821+0000","last_active":"2024-04-25T14:59:47.580772+0000","last_peered":"2024-04-25T14:59:47.580772+0000","last_clean":"2024-04-25T14:59:47.580772+0000","last_became_active":"2024-04-25T14:59:46.299511+0000","last_became_peered":"2024-04-25T14:59:46.299511+0000","last_unstale":"2024-04-25T14:59:47.580772+0000","last_undegraded":"2024-04-25T14:59:47.580772+0000","last_fullsized":"2024-04-25T14:59:47.580772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:23.409333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.290852+0000","last_change":"2024-04-25T14:59:45.331464+0000","last_active":"2024-04-25T14:59:47.290852+0000","last_peered":"2024-04-25T14:59:47.290852+0000","last_clean":"2024-04-25T14:59:47.290852+0000","last_became_active":"2024-04-25T14:59:45.327879+0000","last_became_peered":"2024-04-25T14:59:45.327879+0000","last_unstale":"2024-04-25T14:59:47.290852+0000","last_undegraded":"2024-04-25T14:59:47.290852+0000","last_fullsized":"2024-04-25T14:59:47.290852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:50.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208125+0000","last_change":"2024-04-25T14:59:46.310621+0000","last_active":"2024-04-25T14:59:48.208125+0000","last_peered":"2024-04-25T14:59:48.208125+0000","last_clean":"2024-04-25T14:59:48.208125+0000","last_became_active":"2024-04-25T14:59:46.310292+0000","last_became_peered":"2024-04-25T14:59:46.310292+0000","last_unstale":"2024-04-25T14:59:48.208125+0000","last_undegraded":"2024-04-25T14:59:48.208125+0000","last_fullsized":"2024-04-25T14:59:48.208125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:20.035121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591550+0000","last_change":"2024-04-25T14:59:45.320737+0000","last_active":"2024-04-25T14:59:47.591550+0000","last_peered":"2024-04-25T14:59:47.591550+0000","last_clean":"2024-04-25T14:59:47.591550+0000","last_became_active":"2024-04-25T14:59:45.320404+0000","last_became_peered":"2024-04-25T14:59:45.320404+0000","last_unstale":"2024-04-25T14:59:47.591550+0000","last_undegraded":"2024-04-25T14:59:47.591550+0000","last_fullsized":"2024-04-25T14:59:47.591550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:04.016657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291684+0000","last_change":"2024-04-25T14:59:46.326568+0000","last_active":"2024-04-25T14:59:47.291684+0000","last_peered":"2024-04-25T14:59:47.291684+0000","last_clean":"2024-04-25T14:59:47.291684+0000","last_became_active":"2024-04-25T14:59:46.324019+0000","last_became_peered":"2024-04-25T14:59:46.324019+0000","last_unstale":"2024-04-25T14:59:47.291684+0000","last_undegraded":"2024-04-25T14:59:47.291684+0000","last_fullsized":"2024-04-25T14:59:47.291684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.985449+0000","last_change":"2024-04-25T14:59:45.316935+0000","last_active":"2024-04-25T14:59:47.985449+0000","last_peered":"2024-04-25T14:59:47.985449+0000","last_clean":"2024-04-25T14:59:47.985449+0000","last_became_active":"2024-04-25T14:59:45.316604+0000","last_became_peered":"2024-04-25T14:59:45.316604+0000","last_unstale":"2024-04-25T14:59:47.985449+0000","last_undegraded":"2024-04-25T14:59:47.985449+0000","last_fullsized":"2024-04-25T14:59:47.985449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:45.699926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592778+0000","last_change":"2024-04-25T14:59:46.316195+0000","last_active":"2024-04-25T14:59:47.592778+0000","last_peered":"2024-04-25T14:59:47.592778+0000","last_clean":"2024-04-25T14:59:47.592778+0000","last_became_active":"2024-04-25T14:59:46.315809+0000","last_became_peered":"2024-04-25T14:59:46.315809+0000","last_unstale":"2024-04-25T14:59:47.592778+0000","last_undegraded":"2024-04-25T14:59:47.592778+0000","last_fullsized":"2024-04-25T14:59:47.592778+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:34.440058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591936+0000","last_change":"2024-04-25T14:59:45.325137+0000","last_active":"2024-04-25T14:59:47.591936+0000","last_peered":"2024-04-25T14:59:47.591936+0000","last_clean":"2024-04-25T14:59:47.591936+0000","last_became_active":"2024-04-25T14:59:45.324769+0000","last_became_peered":"2024-04-25T14:59:45.324769+0000","last_unstale":"2024-04-25T14:59:47.591936+0000","last_undegraded":"2024-04-25T14:59:47.591936+0000","last_fullsized":"2024-04-25T14:59:47.591936+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:02.778853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593255+0000","last_change":"2024-04-25T14:59:46.304099+0000","last_active":"2024-04-25T14:59:47.593255+0000","last_peered":"2024-04-25T14:59:47.593255+0000","last_clean":"2024-04-25T14:59:47.593255+0000","last_became_active":"2024-04-25T14:59:46.303774+0000","last_became_peered":"2024-04-25T14:59:46.303774+0000","last_unstale":"2024-04-25T14:59:47.593255+0000","last_undegraded":"2024-04-25T14:59:47.593255+0000","last_fullsized":"2024-04-25T14:59:47.593255+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.308212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291545+0000","last_change":"2024-04-25T14:59:45.322324+0000","last_active":"2024-04-25T14:59:47.291545+0000","last_peered":"2024-04-25T14:59:47.291545+0000","last_clean":"2024-04-25T14:59:47.291545+0000","last_became_active":"2024-04-25T14:59:45.322031+0000","last_became_peered":"2024-04-25T14:59:45.322031+0000","last_unstale":"2024-04-25T14:59:47.291545+0000","last_undegraded":"2024-04-25T14:59:47.291545+0000","last_fullsized":"2024-04-25T14:59:47.291545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.424928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592648+0000","last_change":"2024-04-25T14:59:46.315013+0000","last_active":"2024-04-25T14:59:47.592648+0000","last_peered":"2024-04-25T14:59:47.592648+0000","last_clean":"2024-04-25T14:59:47.592648+0000","last_became_active":"2024-04-25T14:59:46.314654+0000","last_became_peered":"2024-04-25T14:59:46.314654+0000","last_unstale":"2024-04-25T14:59:47.592648+0000","last_undegraded":"2024-04-25T14:59:47.592648+0000","last_fullsized":"2024-04-25T14:59:47.592648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:43.999674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594681+0000","last_change":"2024-04-25T14:59:45.335502+0000","last_active":"2024-04-25T14:59:47.594681+0000","last_peered":"2024-04-25T14:59:47.594681+0000","last_clean":"2024-04-25T14:59:47.594681+0000","last_became_active":"2024-04-25T14:59:45.335018+0000","last_became_peered":"2024-04-25T14:59:45.335018+0000","last_unstale":"2024-04-25T14:59:47.594681+0000","last_undegraded":"2024-04-25T14:59:47.594681+0000","last_fullsized":"2024-04-25T14:59:47.594681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:11.918001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.593390+0000","last_change":"2024-04-25T14:59:46.304766+0000","last_active":"2024-04-25T14:59:47.593390+0000","last_peered":"2024-04-25T14:59:47.593390+0000","last_clean":"2024-04-25T14:59:47.593390+0000","last_became_active":"2024-04-25T14:59:46.304413+0000","last_became_peered":"2024-04-25T14:59:46.304413+0000","last_unstale":"2024-04-25T14:59:47.593390+0000","last_undegraded":"2024-04-25T14:59:47.593390+0000","last_fullsized":"2024-04-25T14:59:47.593390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208504+0000","last_change":"2024-04-25T14:59:46.305285+0000","last_active":"2024-04-25T14:59:48.208504+0000","last_peered":"2024-04-25T14:59:48.208504+0000","last_clean":"2024-04-25T14:59:48.208504+0000","last_became_active":"2024-04-25T14:59:46.304934+0000","last_became_peered":"2024-04-25T14:59:46.304934+0000","last_unstale":"2024-04-25T14:59:48.208504+0000","last_undegraded":"2024-04-25T14:59:48.208504+0000","last_fullsized":"2024-04-25T14:59:48.208504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:44.892921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591714+0000","last_change":"2024-04-25T14:59:46.303263+0000","last_active":"2024-04-25T14:59:47.591714+0000","last_peered":"2024-04-25T14:59:47.591714+0000","last_clean":"2024-04-25T14:59:47.591714+0000","last_became_active":"2024-04-25T14:59:46.302677+0000","last_became_peered":"2024-04-25T14:59:46.302677+0000","last_unstale":"2024-04-25T14:59:47.591714+0000","last_undegraded":"2024-04-25T14:59:47.591714+0000","last_fullsized":"2024-04-25T14:59:47.591714+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:12.792516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.591629+0000","last_change":"2024-04-25T14:59:46.319694+0000","last_active":"2024-04-25T14:59:47.591629+0000","last_peered":"2024-04-25T14:59:47.591629+0000","last_clean":"2024-04-25T14:59:47.591629+0000","last_became_active":"2024-04-25T14:59:46.319417+0000","last_became_peered":"2024-04-25T14:59:46.319417+0000","last_unstale":"2024-04-25T14:59:47.591629+0000","last_undegraded":"2024-04-25T14:59:47.591629+0000","last_fullsized":"2024-04-25T14:59:47.591629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.177498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.528907+0000","last_change":"2024-04-25T14:59:46.325705+0000","last_active":"2024-04-25T14:59:47.528907+0000","last_peered":"2024-04-25T14:59:47.528907+0000","last_clean":"2024-04-25T14:59:47.528907+0000","last_became_active":"2024-04-25T14:59:46.325386+0000","last_became_peered":"2024-04-25T14:59:46.325386+0000","last_unstale":"2024-04-25T14:59:47.528907+0000","last_undegraded":"2024-04-25T14:59:47.528907+0000","last_fullsized":"2024-04-25T14:59:47.528907+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:13:11.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594782+0000","last_change":"2024-04-25T14:59:46.319167+0000","last_active":"2024-04-25T14:59:47.594782+0000","last_peered":"2024-04-25T14:59:47.594782+0000","last_clean":"2024-04-25T14:59:47.594782+0000","last_became_active":"2024-04-25T14:59:46.318804+0000","last_became_peered":"2024-04-25T14:59:46.318804+0000","last_unstale":"2024-04-25T14:59:47.594782+0000","last_undegraded":"2024-04-25T14:59:47.594782+0000","last_fullsized":"2024-04-25T14:59:47.594782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:34.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208294+0000","last_change":"2024-04-25T14:59:46.304554+0000","last_active":"2024-04-25T14:59:48.208294+0000","last_peered":"2024-04-25T14:59:48.208294+0000","last_clean":"2024-04-25T14:59:48.208294+0000","last_became_active":"2024-04-25T14:59:46.304062+0000","last_became_peered":"2024-04-25T14:59:46.304062+0000","last_unstale":"2024-04-25T14:59:48.208294+0000","last_undegraded":"2024-04-25T14:59:48.208294+0000","last_fullsized":"2024-04-25T14:59:48.208294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592222+0000","last_change":"2024-04-25T14:59:46.299104+0000","last_active":"2024-04-25T14:59:47.592222+0000","last_peered":"2024-04-25T14:59:47.592222+0000","last_clean":"2024-04-25T14:59:47.592222+0000","last_became_active":"2024-04-25T14:59:46.298765+0000","last_became_peered":"2024-04-25T14:59:46.298765+0000","last_unstale":"2024-04-25T14:59:47.592222+0000","last_undegraded":"2024-04-25T14:59:47.592222+0000","last_fullsized":"2024-04-25T14:59:47.592222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.494298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291455+0000","last_change":"2024-04-25T14:59:46.328885+0000","last_active":"2024-04-25T14:59:47.291455+0000","last_peered":"2024-04-25T14:59:47.291455+0000","last_clean":"2024-04-25T14:59:47.291455+0000","last_became_active":"2024-04-25T14:59:46.328375+0000","last_became_peered":"2024-04-25T14:59:46.328375+0000","last_unstale":"2024-04-25T14:59:47.291455+0000","last_undegraded":"2024-04-25T14:59:47.291455+0000","last_fullsized":"2024-04-25T14:59:47.291455+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:08.959045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291358+0000","last_change":"2024-04-25T14:59:46.317403+0000","last_active":"2024-04-25T14:59:47.291358+0000","last_peered":"2024-04-25T14:59:47.291358+0000","last_clean":"2024-04-25T14:59:47.291358+0000","last_became_active":"2024-04-25T14:59:46.317100+0000","last_became_peered":"2024-04-25T14:59:46.317100+0000","last_unstale":"2024-04-25T14:59:47.291358+0000","last_undegraded":"2024-04-25T14:59:47.291358+0000","last_fullsized":"2024-04-25T14:59:47.291358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:23.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.291766+0000","last_change":"2024-04-25T14:59:46.327147+0000","last_active":"2024-04-25T14:59:47.291766+0000","last_peered":"2024-04-25T14:59:47.291766+0000","last_clean":"2024-04-25T14:59:47.291766+0000","last_became_active":"2024-04-25T14:59:46.321857+0000","last_became_peered":"2024-04-25T14:59:46.321857+0000","last_unstale":"2024-04-25T14:59:47.291766+0000","last_undegraded":"2024-04-25T14:59:47.291766+0000","last_fullsized":"2024-04-25T14:59:47.291766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:46.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.592139+0000","last_change":"2024-04-25T14:59:46.310841+0000","last_active":"2024-04-25T14:59:47.592139+0000","last_peered":"2024-04-25T14:59:47.592139+0000","last_clean":"2024-04-25T14:59:47.592139+0000","last_became_active":"2024-04-25T14:59:46.310446+0000","last_became_peered":"2024-04-25T14:59:46.310446+0000","last_unstale":"2024-04-25T14:59:47.592139+0000","last_undegraded":"2024-04-25T14:59:47.592139+0000","last_fullsized":"2024-04-25T14:59:47.592139+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.594784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:47.594789+0000","last_change":"2024-04-25T14:59:46.320715+0000","last_active":"2024-04-25T14:59:47.594789+0000","last_peered":"2024-04-25T14:59:47.594789+0000","last_clean":"2024-04-25T14:59:47.594789+0000","last_became_active":"2024-04-25T14:59:46.320238+0000","last_became_peered":"2024-04-25T14:59:46.320238+0000","last_unstale":"2024-04-25T14:59:47.594789+0000","last_undegraded":"2024-04-25T14:59:47.594789+0000","last_fullsized":"2024-04-25T14:59:47.594789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:55.387814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T14:59:48.208192+0000","last_change":"2024-04-25T14:59:46.303772+0000","last_active":"2024-04-25T14:59:48.208192+0000","last_peered":"2024-04-25T14:59:48.208192+0000","last_clean":"2024-04-25T14:59:48.208192+0000","last_became_active":"2024-04-25T14:59:46.303435+0000","last_became_peered":"2024-04-25T14:59:46.303435+0000","last_unstale":"2024-04-25T14:59:48.208192+0000","last_undegraded":"2024-04-25T14:59:48.208192+0000","last_fullsized":"2024-04-25T14:59:48.208192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:06.692979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9010,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":48,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":20430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1540096,"data_stored":1033746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":925696,"data_stored":442154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037790,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":442176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299426,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28504,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716648,"statfs":{"total":95995035648,"available":95965847552,"internally_reserved":0,"allocated":1519616,"data_stored":1032522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":921600,"data_stored":441572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":937984,"data_stored":449784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":441662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":937984,"data_stored":443218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149753,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":441676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888117,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":441594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1032122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":8406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T14:59:54.948 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T14:59:54.949 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T14:59:54.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T14:59:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:54 smithi138 ceph-mon[32875]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:54 smithi138 ceph-mon[32875]: pgmap v312: 97 pgs: 97 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T14:59:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:54 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2388703996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:54 smithi137 ceph-mon[32439]: from='client.24694 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T14:59:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:54 smithi137 ceph-mon[32439]: pgmap v312: 97 pgs: 97 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T14:59:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:54 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2388703996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T14:59:55.429 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T14:59:55.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:55 smithi138 ceph-mon[32875]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:59:55.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:55 smithi094 ceph-mon[27862]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:59:55.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:55 smithi137 ceph-mon[32439]: from='client.14595 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:59:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:56 smithi138 ceph-mon[32875]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:59:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:56 smithi138 ceph-mon[32875]: pgmap v313: 97 pgs: 97 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-04-25T14:59:56.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:56 smithi094 ceph-mon[27862]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:59:56.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:56 smithi094 ceph-mon[27862]: pgmap v313: 97 pgs: 97 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-04-25T14:59:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:56 smithi137 ceph-mon[32439]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T14:59:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:56 smithi137 ceph-mon[32439]: pgmap v313: 97 pgs: 97 active+clean; 585 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 6 op/s 2024-04-25T14:59:57.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 -- 172.21.15.94:0/4229324567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce20076a70 msgr2=0x7fce20074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 --2- 172.21.15.94:0/4229324567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce20076a70 0x7fce20074ed0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fce14009990 tx=0x7fce14009ca0 comp rx=0 tx=0).stop 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 -- 172.21.15.94:0/4229324567 shutdown_connections 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 --2- 172.21.15.94:0/4229324567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce2010f0a0 0x7fce20111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 --2- 172.21.15.94:0/4229324567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce20076a70 0x7fce20074ed0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 --2- 172.21.15.94:0/4229324567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 0x7fce20075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 -- 172.21.15.94:0/4229324567 >> 172.21.15.94:0/4229324567 conn(0x7fce200fe060 msgr2=0x7fce201004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 -- 172.21.15.94:0/4229324567 shutdown_connections 2024-04-25T14:59:57.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.869+0000 7fce28600700 1 -- 172.21.15.94:0/4229324567 wait complete. 2024-04-25T14:59:57.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 Processor -- start 2024-04-25T14:59:57.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 -- start start 2024-04-25T14:59:57.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 0x7fce2019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce20076a70 0x7fce2019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2010f0a0 0x7fce201a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce20114b50 con 0x7fce2010f0a0 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce201149d0 con 0x7fce20076a70 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce28600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce20114cd0 con 0x7fce200754a0 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce25c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 0x7fce2019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce25c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 0x7fce2019f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60430/0 (socket says 172.21.15.94:60430) 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.871+0000 7fce25c00700 1 -- 172.21.15.94:0/2250864656 learned_addr learned my addr 172.21.15.94:0/2250864656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce25200700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce20076a70 0x7fce2019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce26600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2010f0a0 0x7fce201a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce25c00700 1 -- 172.21.15.94:0/2250864656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce20076a70 msgr2=0x7fce2019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce25c00700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce20076a70 0x7fce2019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:57.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce25c00700 1 -- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2010f0a0 msgr2=0x7fce201a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce25c00700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2010f0a0 0x7fce201a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce25c00700 1 -- 172.21.15.94:0/2250864656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce201a6dc0 con 0x7fce200754a0 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce26600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2010f0a0 0x7fce201a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.872+0000 7fce25c00700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 0x7fce2019f160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce140059d0 tx=0x7fce140047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.873+0000 7fce1e000700 1 -- 172.21.15.94:0/2250864656 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14027070 con 0x7fce200754a0 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.873+0000 7fce1e000700 1 -- 172.21.15.94:0/2250864656 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce1400fb80 con 0x7fce200754a0 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.873+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce201a7050 con 0x7fce200754a0 2024-04-25T14:59:57.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.873+0000 7fce1e000700 1 -- 172.21.15.94:0/2250864656 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce14017650 con 0x7fce200754a0 2024-04-25T14:59:57.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.873+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce201a7500 con 0x7fce200754a0 2024-04-25T14:59:57.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.874+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce20050f80 con 0x7fce200754a0 2024-04-25T14:59:57.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.876+0000 7fce1e000700 1 -- 172.21.15.94:0/2250864656 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fce14009d10 con 0x7fce200754a0 2024-04-25T14:59:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.877+0000 7fce1e000700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fce080a3e00 0x7fce080a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T14:59:57.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.877+0000 7fce1e000700 1 -- 172.21.15.94:0/2250864656 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fce140d10e0 con 0x7fce200754a0 2024-04-25T14:59:57.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.877+0000 7fce25200700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fce080a3e00 0x7fce080a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T14:59:57.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.878+0000 7fce25200700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fce080a3e00 0x7fce080a62c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce200767a0 tx=0x7fce0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T14:59:57.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:57.882+0000 7fce1e000700 1 -- 172.21.15.94:0/2250864656 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce14013070 con 0x7fce200754a0 2024-04-25T14:59:58.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.209+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fce2002df80 con 0x7fce200754a0 2024-04-25T14:59:58.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.218+0000 7fce1e000700 1 -- 172.21.15.94:0/2250864656 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fce1409daa0 con 0x7fce200754a0 2024-04-25T14:59:58.220 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-04-25T14:59:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDubypm6MetDBAA+WAEZyvydx0OE3a4hEOT/Q== 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fce080a3e00 msgr2=0x7fce080a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fce080a3e00 0x7fce080a62c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fce200767a0 tx=0x7fce0c009260 comp rx=0 tx=0).stop 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 msgr2=0x7fce2019f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 0x7fce2019f160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce140059d0 tx=0x7fce140047b0 comp rx=0 tx=0).stop 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 shutdown_connections 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fce080a3e00 0x7fce080a62c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce2010f0a0 0x7fce201a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:58.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce200754a0 0x7fce2019f160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:58.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 --2- 172.21.15.94:0/2250864656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce20076a70 0x7fce2019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T14:59:58.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.221+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 >> 172.21.15.94:0/2250864656 conn(0x7fce200fe060 msgr2=0x7fce20102440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T14:59:58.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.222+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 shutdown_connections 2024-04-25T14:59:58.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T14:59:58.222+0000 7fce28600700 1 -- 172.21.15.94:0/2250864656 wait complete. 2024-04-25T14:59:58.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:58 smithi138 ceph-mon[32875]: pgmap v314: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-04-25T14:59:58.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:58 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2250864656' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:59:58.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:58 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:59:58.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 14:59:58 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:59:58.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:58 smithi094 ceph-mon[27862]: pgmap v314: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-04-25T14:59:58.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:58 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2250864656' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:59:58.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:58 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:59:58.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 14:59:58 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:59:58.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:58 smithi137 ceph-mon[32439]: pgmap v314: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2024-04-25T14:59:58.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:58 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2250864656' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:59:58.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:58 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T14:59:58.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 14:59:58 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T14:59:59.099 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T14:59:59.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T14:59:59.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T14:59:59.228 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T14:59:59.229 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T14:59:59.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T14:59:59.715 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:00 smithi138 ceph-mon[32875]: pgmap v315: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T15:00:00.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:00 smithi138 ceph-mon[32875]: overall HEALTH_OK 2024-04-25T15:00:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:00 smithi094 ceph-mon[27862]: pgmap v315: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T15:00:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:00 smithi094 ceph-mon[27862]: overall HEALTH_OK 2024-04-25T15:00:00.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:00 smithi137 ceph-mon[32439]: pgmap v315: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 KiB/s wr, 7 op/s 2024-04-25T15:00:00.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:00 smithi137 ceph-mon[32439]: overall HEALTH_OK 2024-04-25T15:00:01.807 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-25T15:00:01.808 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:01 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-25T15:00:01.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:01 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T15:00:01.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T15:00:01.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-25T15:00:01.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-04-25T15:00:01.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-04-25T15:00:01.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-04-25T15:00:01.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T15:00:01.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-04-25T15:00:01.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-04-25T15:00:01.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:01 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-04-25T15:00:02.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 -- 172.21.15.94:0/161776775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac10e3b0 msgr2=0x7f8aac115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/161776775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac10e3b0 0x7f8aac115f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b0a0 tx=0x7f8aa400b3b0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 -- 172.21.15.94:0/161776775 shutdown_connections 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/161776775 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac10e3b0 0x7f8aac115f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/161776775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 0x7f8aac1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/161776775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aac109c80 0x7f8aac10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 -- 172.21.15.94:0/161776775 >> 172.21.15.94:0/161776775 conn(0x7f8aac076210 msgr2=0x7f8aac107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 -- 172.21.15.94:0/161776775 shutdown_connections 2024-04-25T15:00:02.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.056+0000 7f8ab1c00700 1 -- 172.21.15.94:0/161776775 wait complete. 2024-04-25T15:00:02.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.057+0000 7f8ab1c00700 1 Processor -- start 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.057+0000 7f8ab1c00700 1 -- start start 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.057+0000 7f8ab1c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 0x7f8aac19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8ab1c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac109c80 0x7f8aac19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8ab1c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aac10e3b0 0x7f8aac1a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8ab1c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac1192a0 con 0x7f8aac1092d0 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8ab1c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac119120 con 0x7f8aac109c80 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8ab1c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac119420 con 0x7f8aac10e3b0 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 0x7f8aac19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 0x7f8aac19f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58276/0 (socket says 172.21.15.94:58276) 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaa000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac109c80 0x7f8aac19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 -- 172.21.15.94:0/3476934701 learned_addr learned my addr 172.21.15.94:0/3476934701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:02.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aab400700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aac10e3b0 0x7f8aac1a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 -- 172.21.15.94:0/3476934701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aac10e3b0 msgr2=0x7f8aac1a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aac10e3b0 0x7f8aac1a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 -- 172.21.15.94:0/3476934701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac109c80 msgr2=0x7f8aac19fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac109c80 0x7f8aac19fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.058+0000 7f8aaaa00700 1 -- 172.21.15.94:0/3476934701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac1a4700 con 0x7f8aac1092d0 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.059+0000 7f8aaaa00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 0x7f8aac19f6f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00e9a0 tx=0x7f8a9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.059+0000 7f8aa3400700 1 -- 172.21.15.94:0/3476934701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00ca10 con 0x7f8aac1092d0 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.059+0000 7f8aa3400700 1 -- 172.21.15.94:0/3476934701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a9c00cbb0 con 0x7f8aac1092d0 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.059+0000 7f8aa3400700 1 -- 172.21.15.94:0/3476934701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c018680 con 0x7f8aac1092d0 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.059+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac1a49f0 con 0x7f8aac1092d0 2024-04-25T15:00:02.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.059+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac1ac270 con 0x7f8aac1092d0 2024-04-25T15:00:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.061+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac069730 con 0x7f8aac1092d0 2024-04-25T15:00:02.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.062+0000 7f8aa3400700 1 -- 172.21.15.94:0/3476934701 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8a9c010430 con 0x7f8aac1092d0 2024-04-25T15:00:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.064+0000 7f8aa3400700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8a900a3cd0 0x7f8a900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.064+0000 7f8aa3400700 1 -- 172.21.15.94:0/3476934701 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8240+0+0 (secure 0 0 0) 0x7f8a9c0d0730 con 0x7f8aac1092d0 2024-04-25T15:00:02.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.064+0000 7f8aaa000700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8a900a3cd0 0x7f8a900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:02.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.065+0000 7f8aaa000700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8a900a3cd0 0x7f8a900a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8aac1087c0 tx=0x7f8a98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:02.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.068+0000 7f8aa3400700 1 -- 172.21.15.94:0/3476934701 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a9c09cf70 con 0x7f8aac1092d0 2024-04-25T15:00:02.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.367+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8aac02e140 con 0x7f8aac1092d0 2024-04-25T15:00:02.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.382+0000 7f8aa3400700 1 -- 172.21.15.94:0/3476934701 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f8a9c09c870 con 0x7f8aac1092d0 2024-04-25T15:00:02.389 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:00:02.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8a900a3cd0 msgr2=0x7f8a900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:02.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8a900a3cd0 0x7f8a900a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8aac1087c0 tx=0x7f8a98009260 comp rx=0 tx=0).stop 2024-04-25T15:00:02.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 msgr2=0x7f8aac19f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:02.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 0x7f8aac19f6f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00e9a0 tx=0x7f8a9c00ecb0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 shutdown_connections 2024-04-25T15:00:02.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8a900a3cd0 0x7f8a900a6190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8aac10e3b0 0x7f8aac1a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8aac1092d0 0x7f8aac19f6f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 --2- 172.21.15.94:0/3476934701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8aac109c80 0x7f8aac19fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:02.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 >> 172.21.15.94:0/3476934701 conn(0x7f8aac076210 msgr2=0x7f8aac076750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:02.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 shutdown_connections 2024-04-25T15:00:02.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:02.389+0000 7f8ab1c00700 1 -- 172.21.15.94:0/3476934701 wait complete. 2024-04-25T15:00:02.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: pgmap v316: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-25T15:00:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: pgmap v316: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-25T15:00:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T15:00:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-25T15:00:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T15:00:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T15:00:02.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T15:00:02.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T15:00:02.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-04-25T15:00:02.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-25T15:00:02.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T15:00:02.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T15:00:02.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:02 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3476934701' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:00:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T15:00:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-25T15:00:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T15:00:02.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:02 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3476934701' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:00:02.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: pgmap v316: 97 pgs: 97 active+clean; 620 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.6 KiB/s wr, 5 op/s 2024-04-25T15:00:03.120 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T15:00:03.121 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:02 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3476934701' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:00:03.280 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":2485357518},{"type":"v1","addr":"172.21.15.94:6801","nonce":2485357518}]},"active_addr":"172.21.15.94:6801/2485357518","active_change":"2024-04-25T14:52:01.342856+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":702744035}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":3354212797}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2523030980}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1994260887}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1424873288}]}]} 2024-04-25T15:00:03.285 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T15:00:03.285 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T15:00:03.286 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:00:03.774 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:03 smithi137 ceph-mon[32439]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T15:00:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:03 smithi137 ceph-mon[32439]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.4 KiB/s wr, 6 op/s 2024-04-25T15:00:04.078 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:03 smithi094 ceph-mon[27862]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T15:00:04.078 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:03 smithi094 ceph-mon[27862]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.4 KiB/s wr, 6 op/s 2024-04-25T15:00:04.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:03 smithi138 ceph-mon[32875]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T15:00:04.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:03 smithi138 ceph-mon[32875]: pgmap v319: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.4 KiB/s wr, 6 op/s 2024-04-25T15:00:06.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 -- 172.21.15.94:0/1236848314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd8109c80 msgr2=0x7f3cd810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:06.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1236848314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd8109c80 0x7f3cd810dcd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40099c0 tx=0x7f3cc4009cd0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 -- 172.21.15.94:0/1236848314 shutdown_connections 2024-04-25T15:00:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1236848314 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cd810e3b0 0x7f3cd8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1236848314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 0x7f3cd81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1236848314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd8109c80 0x7f3cd810dcd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 -- 172.21.15.94:0/1236848314 >> 172.21.15.94:0/1236848314 conn(0x7f3cd8076210 msgr2=0x7f3cd8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 -- 172.21.15.94:0/1236848314 shutdown_connections 2024-04-25T15:00:06.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.011+0000 7f3cdd600700 1 -- 172.21.15.94:0/1236848314 wait complete. 2024-04-25T15:00:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 Processor -- start 2024-04-25T15:00:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 -- start start 2024-04-25T15:00:06.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 0x7f3cd819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cd8109c80 0x7f3cd819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd810e3b0 0x7f3cd81a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8119290 con 0x7f3cd81092d0 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8119110 con 0x7f3cd8109c80 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.013+0000 7f3cdd600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd8119410 con 0x7f3cd810e3b0 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 0x7f3cd819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd6a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 0x7f3cd819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58300/0 (socket says 172.21.15.94:58300) 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd6a00700 1 -- 172.21.15.94:0/1014200250 learned_addr learned my addr 172.21.15.94:0/1014200250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd7400700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd810e3b0 0x7f3cd81a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd6000700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cd8109c80 0x7f3cd819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd7400700 1 -- 172.21.15.94:0/1014200250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cd8109c80 msgr2=0x7f3cd819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd7400700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cd8109c80 0x7f3cd819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd7400700 1 -- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 msgr2=0x7f3cd819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:06.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd7400700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 0x7f3cd819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd7400700 1 -- 172.21.15.94:0/1014200250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd81a4730 con 0x7f3cd810e3b0 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd6000700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cd8109c80 0x7f3cd819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd6a00700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 0x7f3cd819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.014+0000 7f3cd7400700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd810e3b0 0x7f3cd81a3ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000efc0 tx=0x7f3cd000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.015+0000 7f3ccf400700 1 -- 172.21.15.94:0/1014200250 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00092e0 con 0x7f3cd810e3b0 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.015+0000 7f3ccf400700 1 -- 172.21.15.94:0/1014200250 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd0004bb0 con 0x7f3cd810e3b0 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.015+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd81a49c0 con 0x7f3cd810e3b0 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.015+0000 7f3ccf400700 1 -- 172.21.15.94:0/1014200250 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0004080 con 0x7f3cd810e3b0 2024-04-25T15:00:06.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.015+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81ac2a0 con 0x7f3cd810e3b0 2024-04-25T15:00:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.016+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd8069730 con 0x7f3cd810e3b0 2024-04-25T15:00:06.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.018+0000 7f3ccf400700 1 -- 172.21.15.94:0/1014200250 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3cd001b030 con 0x7f3cd810e3b0 2024-04-25T15:00:06.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.019+0000 7f3ccf400700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3cbc0a3d50 0x7f3cbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:06.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.019+0000 7f3ccf400700 1 -- 172.21.15.94:0/1014200250 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f3cd00d03b0 con 0x7f3cd810e3b0 2024-04-25T15:00:06.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.020+0000 7f3cd6a00700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3cbc0a3d50 0x7f3cbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:06.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.020+0000 7f3cd6a00700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3cbc0a3d50 0x7f3cbc0a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8011550 tx=0x7f3cc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:06.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.022+0000 7f3ccf400700 1 -- 172.21.15.94:0/1014200250 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cd0022020 con 0x7f3cd810e3b0 2024-04-25T15:00:06.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.248+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cd802df80 con 0x7f3cd810e3b0 2024-04-25T15:00:06.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.250+0000 7f3ccf400700 1 -- 172.21.15.94:0/1014200250 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7f3cd0010040 con 0x7f3cd810e3b0 2024-04-25T15:00:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:00:06.254 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:00:06.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3cbc0a3d50 msgr2=0x7f3cbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:06.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3cbc0a3d50 0x7f3cbc0a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8011550 tx=0x7f3cc8009260 comp rx=0 tx=0).stop 2024-04-25T15:00:06.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd810e3b0 msgr2=0x7f3cd81a3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:06.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd810e3b0 0x7f3cd81a3ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000efc0 tx=0x7f3cd000c3a0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 shutdown_connections 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3cbc0a3d50 0x7f3cbc0a6210 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3cd810e3b0 0x7f3cd81a3ff0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3cd81092d0 0x7f3cd819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 --2- 172.21.15.94:0/1014200250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3cd8109c80 0x7f3cd819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 >> 172.21.15.94:0/1014200250 conn(0x7f3cd8076210 msgr2=0x7f3cd81086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 shutdown_connections 2024-04-25T15:00:06.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:06.254+0000 7f3cdd600700 1 -- 172.21.15.94:0/1014200250 wait complete. 2024-04-25T15:00:06.910 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:06 smithi138 ceph-mon[32875]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:06.911 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:06 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1014200250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:00:06.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:06 smithi094 ceph-mon[27862]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:06.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:06 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1014200250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:00:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:06 smithi137 ceph-mon[32439]: pgmap v320: 97 pgs: 2 peering, 95 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 2 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:06 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1014200250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:00:07.228 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T15:00:07.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:00:07.680 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:08.864 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:08 smithi094 ceph-mon[27862]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:08 smithi138 ceph-mon[32875]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:08 smithi137 ceph-mon[32439]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:09.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.840+0000 7fc4a8a00700 1 -- 172.21.15.94:0/3215352460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00bb00 con 0x7fc4ac074c70 2024-04-25T15:00:09.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.840+0000 7fc4b0e00700 1 -- 172.21.15.94:0/3215352460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac074c70 msgr2=0x7fc4ac075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:09.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.840+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/3215352460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac074c70 0x7fc4ac075050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009990 tx=0x7fc49c009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.841+0000 7fc4b0e00700 1 -- 172.21.15.94:0/3215352460 shutdown_connections 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.841+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/3215352460 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ac113c10 0x7fc4ac116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.841+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/3215352460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac074c70 0x7fc4ac075050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.841+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/3215352460 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 0x7fc4ac1136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.841+0000 7fc4b0e00700 1 -- 172.21.15.94:0/3215352460 >> 172.21.15.94:0/3215352460 conn(0x7fc4ac079ba0 msgr2=0x7fc4ac079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.841+0000 7fc4b0e00700 1 -- 172.21.15.94:0/3215352460 shutdown_connections 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.841+0000 7fc4b0e00700 1 -- 172.21.15.94:0/3215352460 wait complete. 2024-04-25T15:00:09.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.842+0000 7fc4b0e00700 1 Processor -- start 2024-04-25T15:00:09.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.842+0000 7fc4b0e00700 1 -- start start 2024-04-25T15:00:09.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.842+0000 7fc4b0e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ac074c70 0x7fc4ac19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:09.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.842+0000 7fc4b0e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 0x7fc4ac19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:09.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.842+0000 7fc4b0e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac113c10 0x7fc4ac1a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:09.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4b0e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac119220 con 0x7fc4ac113c10 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4b0e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1190a0 con 0x7fc4ac074c70 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4b0e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ac1193a0 con 0x7fc4ac075590 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4aa800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac113c10 0x7fc4ac1a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 0x7fc4ac19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ac074c70 0x7fc4ac19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ac074c70 0x7fc4ac19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:52464/0 (socket says 172.21.15.94:52464) 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 0x7fc4ac19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40958/0 (socket says 172.21.15.94:40958) 2024-04-25T15:00:09.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9e00700 1 -- 172.21.15.94:0/2571040745 learned_addr learned my addr 172.21.15.94:0/2571040745 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9400700 1 -- 172.21.15.94:0/2571040745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ac074c70 msgr2=0x7fc4ac19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9400700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ac074c70 0x7fc4ac19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9400700 1 -- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac113c10 msgr2=0x7fc4ac1a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9400700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac113c10 0x7fc4ac1a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.843+0000 7fc4a9400700 1 -- 172.21.15.94:0/2571040745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ac1a4760 con 0x7fc4ac075590 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.844+0000 7fc4a9400700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 0x7fc4ac19fc90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc49800e9f0 tx=0x7fc49800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.844+0000 7fc4a2a00700 1 -- 172.21.15.94:0/2571040745 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498004220 con 0x7fc4ac075590 2024-04-25T15:00:09.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.844+0000 7fc4a2a00700 1 -- 172.21.15.94:0/2571040745 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4980043c0 con 0x7fc4ac075590 2024-04-25T15:00:09.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.844+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ac1a4a50 con 0x7fc4ac075590 2024-04-25T15:00:09.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.844+0000 7fc4a2a00700 1 -- 172.21.15.94:0/2571040745 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498005020 con 0x7fc4ac075590 2024-04-25T15:00:09.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.844+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ac1ac2d0 con 0x7fc4ac075590 2024-04-25T15:00:09.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.846+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ac069730 con 0x7fc4ac075590 2024-04-25T15:00:09.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.847+0000 7fc4a2a00700 1 -- 172.21.15.94:0/2571040745 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc498004750 con 0x7fc4ac075590 2024-04-25T15:00:09.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.849+0000 7fc4a2a00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc4900a3d50 0x7fc4900a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:09.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.849+0000 7fc4a2a00700 1 -- 172.21.15.94:0/2571040745 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc4980d0480 con 0x7fc4ac075590 2024-04-25T15:00:09.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.849+0000 7fc4a9e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc4900a3d50 0x7fc4900a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:09.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.850+0000 7fc4a9e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc4900a3d50 0x7fc4900a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b2e0 tx=0x7fc49c009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:09.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:09.853+0000 7fc4a2a00700 1 -- 172.21.15.94:0/2571040745 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc49809ccc0 con 0x7fc4ac075590 2024-04-25T15:00:10.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.079+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4ac1ac560 con 0x7fc4ac075590 2024-04-25T15:00:10.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.081+0000 7fc4a2a00700 1 -- 172.21.15.94:0/2571040745 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7fc49809c5c0 con 0x7fc4ac075590 2024-04-25T15:00:10.083 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:00:10.083 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:00:10.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.084+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc4900a3d50 msgr2=0x7fc4900a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:10.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.084+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc4900a3d50 0x7fc4900a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00b2e0 tx=0x7fc49c009510 comp rx=0 tx=0).stop 2024-04-25T15:00:10.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.084+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 msgr2=0x7fc4ac19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.084+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 0x7fc4ac19fc90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc49800e9f0 tx=0x7fc49800ed00 comp rx=0 tx=0).stop 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 shutdown_connections 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc4900a3d50 0x7fc4900a6210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4ac113c10 0x7fc4ac1a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc4ac074c70 0x7fc4ac19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 --2- 172.21.15.94:0/2571040745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac075590 0x7fc4ac19fc90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 >> 172.21.15.94:0/2571040745 conn(0x7fc4ac079ba0 msgr2=0x7fc4ac112ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 shutdown_connections 2024-04-25T15:00:10.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:10.085+0000 7fc4b0e00700 1 -- 172.21.15.94:0/2571040745 wait complete. 2024-04-25T15:00:10.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:10 smithi094 ceph-mon[27862]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:10.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:10 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2571040745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:00:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:10 smithi138 ceph-mon[32875]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:10 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2571040745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:00:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:10 smithi137 ceph-mon[32439]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.4 KiB/s wr, 3 op/s; 14 B/s, 0 objects/s recovering 2024-04-25T15:00:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:10 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2571040745' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:00:11.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T15:00:11.020 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T15:00:11.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T15:00:11.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T15:00:11.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T15:00:11.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T15:00:11.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T15:00:11.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T15:00:11.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T15:00:11.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T15:00:11.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T15:00:11.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T15:00:12.080 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:12.089 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:12.098 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:12.101 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:12.795 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:12 smithi094 ceph-mon[27862]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 523 B/s rd, 1 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T15:00:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:12 smithi138 ceph-mon[32875]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 523 B/s rd, 1 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T15:00:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:12 smithi137 ceph-mon[32439]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 523 B/s rd, 1 op/s; 11 B/s, 0 objects/s recovering 2024-04-25T15:00:14.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:14 smithi094 ceph-mon[27862]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 285 B/s wr, 2 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T15:00:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:14 smithi138 ceph-mon[32875]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 285 B/s wr, 2 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T15:00:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:14 smithi137 ceph-mon[32439]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 285 B/s wr, 2 op/s; 10 B/s, 0 objects/s recovering 2024-04-25T15:00:15.386 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:15.389 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:15.391 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:15.394 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:15.410 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:16.861 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:16 smithi094 ceph-mon[27862]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T15:00:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:16 smithi138 ceph-mon[32875]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T15:00:16.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:16 smithi137 ceph-mon[32439]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s; 9 B/s, 0 objects/s recovering 2024-04-25T15:00:18.844 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:18 smithi094 ceph-mon[27862]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T15:00:18.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:18 smithi138 ceph-mon[32875]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T15:00:18.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:18 smithi137 ceph-mon[32439]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T15:00:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.005+0000 7f504c000700 1 -- 172.21.15.94:0/2413319900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5044105870 msgr2=0x7f50441098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.005+0000 7f504c000700 1 --2- 172.21.15.94:0/2413319900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5044105870 0x7f50441098c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5034009990 tx=0x7f5034009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.006+0000 7f504c000700 1 -- 172.21.15.94:0/2413319900 shutdown_connections 2024-04-25T15:00:19.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.006+0000 7f504c000700 1 --2- 172.21.15.94:0/2413319900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5044109fa0 0x7f5044111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.006+0000 7f504c000700 1 --2- 172.21.15.94:0/2413319900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5044104ec0 0x7f50441052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.006+0000 7f504c000700 1 --2- 172.21.15.94:0/2413319900 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5044105870 0x7f50441098c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.006+0000 7f504c000700 1 -- 172.21.15.94:0/2413319900 >> 172.21.15.94:0/2413319900 conn(0x7f5044100750 msgr2=0x7f5044102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.006+0000 7f504c000700 1 -- 172.21.15.94:0/2413319900 shutdown_connections 2024-04-25T15:00:19.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.006+0000 7f504c000700 1 -- 172.21.15.94:0/2413319900 wait complete. 2024-04-25T15:00:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 Processor -- start 2024-04-25T15:00:19.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 -- start start 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5044104ec0 0x7f504419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5044105870 0x7f504419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5044109fa0 0x7f504419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5044114e10 con 0x7f5044109fa0 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5044114c90 con 0x7f5044105870 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f504c000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5044114f90 con 0x7f5044104ec0 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f5049600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5044104ec0 0x7f504419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.007+0000 7f5049600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5044104ec0 0x7f504419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:47078/0 (socket says 172.21.15.94:47078) 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5049600700 1 -- 172.21.15.94:0/713757857 learned_addr learned my addr 172.21.15.94:0/713757857 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f504a000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5044109fa0 0x7f504419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5048c00700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5044105870 0x7f504419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:19.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5049600700 1 -- 172.21.15.94:0/713757857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5044105870 msgr2=0x7f504419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5049600700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5044105870 0x7f504419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5049600700 1 -- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5044109fa0 msgr2=0x7f504419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5049600700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5044109fa0 0x7f504419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5049600700 1 -- 172.21.15.94:0/713757857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50441a02e0 con 0x7f5044104ec0 2024-04-25T15:00:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f504a000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5044109fa0 0x7f504419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:19.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.008+0000 7f5049600700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5044104ec0 0x7f504419b320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f503800eb30 tx=0x7f503800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.009+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503800cbd0 con 0x7f5044104ec0 2024-04-25T15:00:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.009+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50380044d0 con 0x7f5044104ec0 2024-04-25T15:00:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.009+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038003e30 con 0x7f5044104ec0 2024-04-25T15:00:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.009+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50441a05d0 con 0x7f5044104ec0 2024-04-25T15:00:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.009+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50441a7e50 con 0x7f5044104ec0 2024-04-25T15:00:19.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.010+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5044063150 con 0x7f5044104ec0 2024-04-25T15:00:19.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.012+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5038004000 con 0x7f5044104ec0 2024-04-25T15:00:19.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.013+0000 7f5042000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f502c0a3cd0 0x7f502c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.013+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f50380d0d50 con 0x7f5044104ec0 2024-04-25T15:00:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.014+0000 7f5048c00700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f502c0a3cd0 0x7f502c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.014+0000 7f5042000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] conn(0x7f502c0ad890 0x7f502c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:19.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.014+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 --> [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5038017e50 con 0x7f502c0ad890 2024-04-25T15:00:19.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.014+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f50380d1100 con 0x7f5044104ec0 2024-04-25T15:00:19.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.014+0000 7f504a000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] conn(0x7f502c0ad890 0x7f502c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:19.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.014+0000 7f5048c00700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f502c0a3cd0 0x7f502c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5034000c00 tx=0x7f5034005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:19.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.015+0000 7f504a000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] conn(0x7f502c0ad890 0x7f502c0afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:19.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.016+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== osd.3 v2:172.21.15.94:6810/462961415 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5038017e50 con 0x7f502c0ad890 2024-04-25T15:00:19.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.033+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 --> [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f50440635d0 con 0x7f502c0ad890 2024-04-25T15:00:19.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.034+0000 7f5042000700 1 -- 172.21.15.94:0/713757857 <== osd.3 v2:172.21.15.94:6810/462961415 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f50440635d0 con 0x7f502c0ad890 2024-04-25T15:00:19.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] conn(0x7f502c0ad890 msgr2=0x7f502c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:19.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] conn(0x7f502c0ad890 0x7f502c0afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:19.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f502c0a3cd0 msgr2=0x7f502c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:19.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f502c0a3cd0 0x7f502c0a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5034000c00 tx=0x7f5034005960 comp rx=0 tx=0).stop 2024-04-25T15:00:19.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5044104ec0 msgr2=0x7f504419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:19.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 --2- 172.21.15.94:0/713757857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5044104ec0 0x7f504419b320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f503800eb30 tx=0x7f503800ee40 comp rx=0 tx=0).stop 2024-04-25T15:00:19.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504a000700 1 -- 172.21.15.94:0/713757857 reap_dead start 2024-04-25T15:00:19.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 shutdown_connections 2024-04-25T15:00:19.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 >> 172.21.15.94:0/713757857 conn(0x7f5044100750 msgr2=0x7f5044102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:19.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 shutdown_connections 2024-04-25T15:00:19.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:19.035+0000 7f504c000700 1 -- 172.21.15.94:0/713757857 wait complete. 2024-04-25T15:00:20.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:20 smithi094 ceph-mon[27862]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:00:20.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:20 smithi138 ceph-mon[32875]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:00:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:20 smithi137 ceph-mon[32439]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:00:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.376+0000 7f6053400700 1 -- 172.21.15.94:0/2372443787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f604c06a750 msgr2=0x7f604c06ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.376+0000 7f6053400700 1 --2- 172.21.15.94:0/2372443787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f604c06a750 0x7f604c06ab30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6040009990 tx=0x7f6040009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.376+0000 7f6053400700 1 -- 172.21.15.94:0/2372443787 shutdown_connections 2024-04-25T15:00:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.376+0000 7f6053400700 1 --2- 172.21.15.94:0/2372443787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604c10f4f0 0x7f604c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.376+0000 7f6053400700 1 --2- 172.21.15.94:0/2372443787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f604c06a750 0x7f604c06ab30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.376+0000 7f6053400700 1 --2- 172.21.15.94:0/2372443787 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f604c06b070 0x7f604c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.376+0000 7f6053400700 1 -- 172.21.15.94:0/2372443787 >> 172.21.15.94:0/2372443787 conn(0x7f604c0788f0 msgr2=0x7f604c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.377+0000 7f6053400700 1 -- 172.21.15.94:0/2372443787 shutdown_connections 2024-04-25T15:00:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.377+0000 7f6053400700 1 -- 172.21.15.94:0/2372443787 wait complete. 2024-04-25T15:00:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.378+0000 7f6053400700 1 Processor -- start 2024-04-25T15:00:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.379+0000 7f6053400700 1 -- start start 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.379+0000 7f6053400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f604c06a750 0x7f604c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.379+0000 7f6053400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604c06b070 0x7f604c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.379+0000 7f6053400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f604c10f4f0 0x7f604c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.379+0000 7f6053400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f604c114e00 con 0x7f604c06b070 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.379+0000 7f6053400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f604c114c80 con 0x7f604c06a750 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.379+0000 7f6053400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f604c114f80 con 0x7f604c10f4f0 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f604c06a750 0x7f604c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f604be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604c06b070 0x7f604c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f604c06a750 0x7f604c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37566/0 (socket says 172.21.15.94:37566) 2024-04-25T15:00:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 -- 172.21.15.94:0/3290176555 learned_addr learned my addr 172.21.15.94:0/3290176555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:22.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6051400700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f604c10f4f0 0x7f604c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:22.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 -- 172.21.15.94:0/3290176555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f604c10f4f0 msgr2=0x7f604c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:22.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f604c10f4f0 0x7f604c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 -- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604c06b070 msgr2=0x7f604c19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:22.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f604c06b070 0x7f604c19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.380+0000 7f6050a00700 1 -- 172.21.15.94:0/3290176555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604c1a0320 con 0x7f604c06a750 2024-04-25T15:00:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.381+0000 7f6050a00700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f604c06a750 0x7f604c19b310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f604000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.381+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040027070 con 0x7f604c06a750 2024-04-25T15:00:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.381+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6040022470 con 0x7f604c06a750 2024-04-25T15:00:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.381+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040017650 con 0x7f604c06a750 2024-04-25T15:00:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.382+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604c1a05b0 con 0x7f604c06a750 2024-04-25T15:00:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.382+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f604c1a7e90 con 0x7f604c06a750 2024-04-25T15:00:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.383+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f604c1a0a40 con 0x7f604c06a750 2024-04-25T15:00:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.385+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f604002a070 con 0x7f604c06a750 2024-04-25T15:00:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.386+0000 7f6049600700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60340a3cd0 0x7f60340a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.387+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f60400d1950 con 0x7f604c06a750 2024-04-25T15:00:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.387+0000 7f604be00700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60340a3cd0 0x7f60340a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.387+0000 7f6049600700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] conn(0x7f60340ad890 0x7f60340afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.387+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 --> [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6040001d30 con 0x7f60340ad890 2024-04-25T15:00:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.387+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f60400d1d00 con 0x7f604c06a750 2024-04-25T15:00:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.387+0000 7f6051400700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] conn(0x7f60340ad890 0x7f60340afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.387+0000 7f604be00700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60340a3cd0 0x7f60340a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f604c102f00 tx=0x7f6038009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.388+0000 7f6051400700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] conn(0x7f60340ad890 0x7f60340afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:22.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.389+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== osd.0 v2:172.21.15.94:6802/1239034236 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6040001d30 con 0x7f60340ad890 2024-04-25T15:00:22.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.407+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 --> [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f604c063460 con 0x7f60340ad890 2024-04-25T15:00:22.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.408+0000 7f6049600700 1 -- 172.21.15.94:0/3290176555 <== osd.0 v2:172.21.15.94:6802/1239034236 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f604c063460 con 0x7f60340ad890 2024-04-25T15:00:22.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.408+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] conn(0x7f60340ad890 msgr2=0x7f60340afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.408+0000 7f6053400700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] conn(0x7f60340ad890 0x7f60340afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60340a3cd0 msgr2=0x7f60340a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f60340a3cd0 0x7f60340a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f604c102f00 tx=0x7f6038009260 comp rx=0 tx=0).stop 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f604c06a750 msgr2=0x7f604c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 --2- 172.21.15.94:0/3290176555 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f604c06a750 0x7f604c19b310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f604000bfd0 comp rx=0 tx=0).stop 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6051400700 1 -- 172.21.15.94:0/3290176555 reap_dead start 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 shutdown_connections 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 >> 172.21.15.94:0/3290176555 conn(0x7f604c0788f0 msgr2=0x7f604c1109c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 shutdown_connections 2024-04-25T15:00:22.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:22.409+0000 7f6053400700 1 -- 172.21.15.94:0/3290176555 wait complete. 2024-04-25T15:00:22.598 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:22 smithi094 ceph-mon[27862]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:00:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:22 smithi138 ceph-mon[32875]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:00:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:22 smithi137 ceph-mon[32439]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:00:24.144 INFO:teuthology.orchestra.run.smithi094.stdout:111669149759 2024-04-25T15:00:24.144 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T15:00:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:24 smithi094 ceph-mon[27862]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:00:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:24 smithi138 ceph-mon[32875]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:00:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:24 smithi137 ceph-mon[32439]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:00:26.727 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:26 smithi094 ceph-mon[27862]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:26 smithi138 ceph-mon[32875]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:26 smithi137 ceph-mon[32439]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 -- 172.21.15.94:0/823092704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10102af0 msgr2=0x7f0b10102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 --2- 172.21.15.94:0/823092704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10102af0 0x7f0b10102f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009990 tx=0x7f0afc009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 -- 172.21.15.94:0/823092704 shutdown_connections 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 --2- 172.21.15.94:0/823092704 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b101034b0 0x7f0b1010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 --2- 172.21.15.94:0/823092704 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b10107940 0x7f0b10107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 --2- 172.21.15.94:0/823092704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b10102af0 0x7f0b10102f70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 -- 172.21.15.94:0/823092704 >> 172.21.15.94:0/823092704 conn(0x7f0b10079ba0 msgr2=0x7f0b10079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 -- 172.21.15.94:0/823092704 shutdown_connections 2024-04-25T15:00:28.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.725+0000 7f0b18600700 1 -- 172.21.15.94:0/823092704 wait complete. 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.726+0000 7f0b18600700 1 Processor -- start 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b18600700 1 -- start start 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b18600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b10102af0 0x7f0b10199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b18600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b101034b0 0x7f0b10199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b18600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b10107940 0x7f0b1019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b18600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10112c30 con 0x7f0b10102af0 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b18600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10112ab0 con 0x7f0b10107940 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b18600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10112db0 con 0x7f0b101034b0 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b15c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b10102af0 0x7f0b10199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b15c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b10102af0 0x7f0b10199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52858/0 (socket says 172.21.15.94:52858) 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b15c00700 1 -- 172.21.15.94:0/480663984 learned_addr learned my addr 172.21.15.94:0/480663984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b15200700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b101034b0 0x7f0b10199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:28.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.727+0000 7f0b16600700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b10107940 0x7f0b1019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b15c00700 1 -- 172.21.15.94:0/480663984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b101034b0 msgr2=0x7f0b10199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b15c00700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b101034b0 0x7f0b10199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b15c00700 1 -- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b10107940 msgr2=0x7f0b1019d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b15c00700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0b10107940 0x7f0b1019d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b15c00700 1 -- 172.21.15.94:0/480663984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1019e0c0 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b15c00700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b10102af0 0x7f0b10199100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400e9a0 tx=0x7f0b0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400ca10 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0400cbb0 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1019e3b0 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.728+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04018680 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.729+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101a5c30 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.729+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b100631d0 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.731+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0b04010430 con 0x7f0b10102af0 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.733+0000 7f0b0e000700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0af80a3d50 0x7f0af80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:28.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.733+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0b040d0bb0 con 0x7f0b10102af0 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.733+0000 7f0b15200700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0af80a3d50 0x7f0af80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.733+0000 7f0b0e000700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] conn(0x7f0af80ad910 0x7f0af80afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.733+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 --> [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b04019ec0 con 0x7f0af80ad910 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.733+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0b040d0f60 con 0x7f0b10102af0 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.734+0000 7f0b15200700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0af80a3d50 0x7f0af80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b1007a1f0 tx=0x7f0afc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.734+0000 7f0b16600700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] conn(0x7f0af80ad910 0x7f0af80afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.734+0000 7f0b16600700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] conn(0x7f0af80ad910 0x7f0af80afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.735+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== osd.1 v2:172.21.15.137:6800/3278622901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b04019ec0 con 0x7f0af80ad910 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.753+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 --> [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b100635d0 con 0x7f0af80ad910 2024-04-25T15:00:28.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.754+0000 7f0b0e000700 1 -- 172.21.15.94:0/480663984 <== osd.1 v2:172.21.15.137:6800/3278622901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0b100635d0 con 0x7f0af80ad910 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] conn(0x7f0af80ad910 msgr2=0x7f0af80afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] conn(0x7f0af80ad910 0x7f0af80afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0af80a3d50 msgr2=0x7f0af80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0af80a3d50 0x7f0af80a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b1007a1f0 tx=0x7f0afc005960 comp rx=0 tx=0).stop 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b10102af0 msgr2=0x7f0b10199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 --2- 172.21.15.94:0/480663984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0b10102af0 0x7f0b10199100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400e9a0 tx=0x7f0b0400ecb0 comp rx=0 tx=0).stop 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b16600700 1 -- 172.21.15.94:0/480663984 reap_dead start 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 shutdown_connections 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 >> 172.21.15.94:0/480663984 conn(0x7f0b10079ba0 msgr2=0x7f0b10100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 shutdown_connections 2024-04-25T15:00:28.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:26.755+0000 7f0b18600700 1 -- 172.21.15.94:0/480663984 wait complete. 2024-04-25T15:00:28.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:28 smithi094 ceph-mon[27862]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:28.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:28 smithi138 ceph-mon[32875]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:28 smithi137 ceph-mon[32439]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:29.230 INFO:teuthology.orchestra.run.smithi094.stdout:55834574925 2024-04-25T15:00:29.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T15:00:29.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 -- 172.21.15.94:0/4125505420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd568108a30 msgr2=0x7fd568108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:29.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 --2- 172.21.15.94:0/4125505420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd568108a30 0x7fd568108e10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd558009990 tx=0x7fd558009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 -- 172.21.15.94:0/4125505420 shutdown_connections 2024-04-25T15:00:29.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 --2- 172.21.15.94:0/4125505420 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd56806b0f0 0x7fd56810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 --2- 172.21.15.94:0/4125505420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd568108a30 0x7fd568108e10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 --2- 172.21.15.94:0/4125505420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd56806a750 0x7fd56806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 -- 172.21.15.94:0/4125505420 >> 172.21.15.94:0/4125505420 conn(0x7fd5680788f0 msgr2=0x7fd568078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 -- 172.21.15.94:0/4125505420 shutdown_connections 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.493+0000 7fd56d200700 1 -- 172.21.15.94:0/4125505420 wait complete. 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.494+0000 7fd56d200700 1 Processor -- start 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd56d200700 1 -- start start 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd56d200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd56806a750 0x7fd568199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd56d200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd56806b0f0 0x7fd568199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd56d200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd568108a30 0x7fd56819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd56d200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd568112c00 con 0x7fd568108a30 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd56d200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd568112a80 con 0x7fd56806b0f0 2024-04-25T15:00:29.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd56d200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd568112d80 con 0x7fd56806a750 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd56806b0f0 0x7fd568199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd566800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd568108a30 0x7fd56819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd565e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd56806a750 0x7fd568199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd55fe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd56806b0f0 0x7fd568199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:37588/0 (socket says 172.21.15.94:37588) 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.495+0000 7fd55fe00700 1 -- 172.21.15.94:0/2210173025 learned_addr learned my addr 172.21.15.94:0/2210173025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55fe00700 1 -- 172.21.15.94:0/2210173025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd56806a750 msgr2=0x7fd568199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55fe00700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd56806a750 0x7fd568199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55fe00700 1 -- 172.21.15.94:0/2210173025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd568108a30 msgr2=0x7fd56819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55fe00700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd568108a30 0x7fd56819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55fe00700 1 -- 172.21.15.94:0/2210173025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56819e140 con 0x7fd56806b0f0 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55fe00700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd56806b0f0 0x7fd568199670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9a0 tx=0x7fd55000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55000ca10 con 0x7fd56806b0f0 2024-04-25T15:00:29.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56819e430 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55000cbb0 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.496+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550018680 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.497+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5681a5cb0 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.497+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5680631d0 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.500+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd550010430 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.501+0000 7fd55ea00700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd54c0a3d50 0x7fd54c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.501+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd5500d0c40 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.501+0000 7fd565e00700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd54c0a3d50 0x7fd54c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.501+0000 7fd55ea00700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] conn(0x7fd54c0ad910 0x7fd54c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.502+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 --> [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd550019ec0 con 0x7fd54c0ad910 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.502+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd5500d0ff0 con 0x7fd56806b0f0 2024-04-25T15:00:29.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.502+0000 7fd565e00700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd54c0a3d50 0x7fd54c0a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b2e0 tx=0x7fd558005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.502+0000 7fd566800700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] conn(0x7fd54c0ad910 0x7fd54c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.502+0000 7fd566800700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] conn(0x7fd54c0ad910 0x7fd54c0afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.503+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== osd.4 v2:172.21.15.137:6808/2944995107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd550019ec0 con 0x7fd54c0ad910 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.521+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 --> [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5680635d0 con 0x7fd54c0ad910 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd55ea00700 1 -- 172.21.15.94:0/2210173025 <== osd.4 v2:172.21.15.137:6808/2944995107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd5680635d0 con 0x7fd54c0ad910 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] conn(0x7fd54c0ad910 msgr2=0x7fd54c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] conn(0x7fd54c0ad910 0x7fd54c0afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd54c0a3d50 msgr2=0x7fd54c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd54c0a3d50 0x7fd54c0a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b2e0 tx=0x7fd558005fd0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd56806b0f0 msgr2=0x7fd568199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 --2- 172.21.15.94:0/2210173025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd56806b0f0 0x7fd568199670 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd55000e9a0 tx=0x7fd55000ecb0 comp rx=0 tx=0).stop 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd566800700 1 -- 172.21.15.94:0/2210173025 reap_dead start 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 shutdown_connections 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 >> 172.21.15.94:0/2210173025 conn(0x7fd5680788f0 msgr2=0x7fd5680ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 shutdown_connections 2024-04-25T15:00:29.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:24.522+0000 7fd56d200700 1 -- 172.21.15.94:0/2210173025 wait complete. 2024-04-25T15:00:30.533 INFO:teuthology.orchestra.run.smithi094.stdout:34359738448 2024-04-25T15:00:30.534 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T15:00:30.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:30 smithi094 ceph-mon[27862]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:30.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:30 smithi138 ceph-mon[32875]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:30 smithi137 ceph-mon[32439]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.353+0000 7f0ebc400700 1 -- 172.21.15.94:0/2662449193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb41019d0 msgr2=0x7f0eb410f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.353+0000 7f0ebc400700 1 --2- 172.21.15.94:0/2662449193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb41019d0 0x7f0eb410f910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0eac0098f0 tx=0x7f0eac009c00 comp rx=0 tx=0).stop 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.353+0000 7f0ebc400700 1 -- 172.21.15.94:0/2662449193 shutdown_connections 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.353+0000 7f0ebc400700 1 --2- 172.21.15.94:0/2662449193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb41019d0 0x7f0eb410f910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.353+0000 7f0ebc400700 1 --2- 172.21.15.94:0/2662449193 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108ad0 0x7f0eb4108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.353+0000 7f0ebc400700 1 --2- 172.21.15.94:0/2662449193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb4101030 0x7f0eb4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.353+0000 7f0ebc400700 1 -- 172.21.15.94:0/2662449193 >> 172.21.15.94:0/2662449193 conn(0x7f0eb4077fb0 msgr2=0x7f0eb40ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.354+0000 7f0ebc400700 1 -- 172.21.15.94:0/2662449193 shutdown_connections 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.354+0000 7f0ebc400700 1 -- 172.21.15.94:0/2662449193 wait complete. 2024-04-25T15:00:31.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.354+0000 7f0ebc400700 1 Processor -- start 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0ebc400700 1 -- start start 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0ebc400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4101030 0x7f0eb4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0ebc400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb41019d0 0x7f0eb4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0ebc400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108ad0 0x7f0eb419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0ebc400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4112c50 con 0x7f0eb4101030 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0ebc400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4112ad0 con 0x7f0eb4108ad0 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0ebc400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4112dd0 con 0x7f0eb41019d0 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0eb9a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4101030 0x7f0eb4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0eb9a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4101030 0x7f0eb4199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38704/0 (socket says 172.21.15.94:38704) 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0eb9a00700 1 -- 172.21.15.94:0/672937174 learned_addr learned my addr 172.21.15.94:0/672937174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0eb9000700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb41019d0 0x7f0eb4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:31.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.355+0000 7f0eba400700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108ad0 0x7f0eb419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eb9a00700 1 -- 172.21.15.94:0/672937174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb41019d0 msgr2=0x7f0eb4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eb9a00700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb41019d0 0x7f0eb4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eb9a00700 1 -- 172.21.15.94:0/672937174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108ad0 msgr2=0x7f0eb419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eb9a00700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108ad0 0x7f0eb419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eb9a00700 1 -- 172.21.15.94:0/672937174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb419e210 con 0x7f0eb4101030 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eba400700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0eb4108ad0 0x7f0eb419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eb9000700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0eb41019d0 0x7f0eb4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.356+0000 7f0eb9a00700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4101030 0x7f0eb4199200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800ef10 tx=0x7f0ea800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.357+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800c9b0 con 0x7f0eb4101030 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.357+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb419e4a0 con 0x7f0eb4101030 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.357+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea8019070 con 0x7f0eb4101030 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.357+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea8014480 con 0x7f0eb4101030 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.357+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb41a5d80 con 0x7f0eb4101030 2024-04-25T15:00:31.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.358+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0eb4063150 con 0x7f0eb4101030 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.360+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0ea8012050 con 0x7f0eb4101030 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.361+0000 7f0eb2000700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e9c0a3cd0 0x7f0e9c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.361+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0ea80d0f40 con 0x7f0eb4101030 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.361+0000 7f0eb9000700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e9c0a3cd0 0x7f0e9c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.361+0000 7f0eb2000700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] conn(0x7f0e9c0ad890 0x7f0e9c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.361+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 --> [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ea8015ec0 con 0x7f0e9c0ad890 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.361+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0ea80d12f0 con 0x7f0eb4101030 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.362+0000 7f0eba400700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] conn(0x7f0e9c0ad890 0x7f0e9c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.362+0000 7f0eb9000700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e9c0a3cd0 0x7f0e9c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40ff0d0 tx=0x7f0ea0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:31.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.362+0000 7f0eba400700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] conn(0x7f0e9c0ad890 0x7f0e9c0afd30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.363+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== osd.6 v2:172.21.15.94:6818/194684740 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ea8015ec0 con 0x7f0e9c0ad890 2024-04-25T15:00:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.381+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 --> [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0eb4063550 con 0x7f0e9c0ad890 2024-04-25T15:00:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.382+0000 7f0eb2000700 1 -- 172.21.15.94:0/672937174 <== osd.6 v2:172.21.15.94:6818/194684740 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0eb4063550 con 0x7f0e9c0ad890 2024-04-25T15:00:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.382+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] conn(0x7f0e9c0ad890 msgr2=0x7f0e9c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.382+0000 7f0ebc400700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] conn(0x7f0e9c0ad890 0x7f0e9c0afd30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e9c0a3cd0 msgr2=0x7f0e9c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:31.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0e9c0a3cd0 0x7f0e9c0a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40ff0d0 tx=0x7f0ea0009310 comp rx=0 tx=0).stop 2024-04-25T15:00:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4101030 msgr2=0x7f0eb4199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 --2- 172.21.15.94:0/672937174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0eb4101030 0x7f0eb4199200 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800ef10 tx=0x7f0ea800c3a0 comp rx=0 tx=0).stop 2024-04-25T15:00:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0eba400700 1 -- 172.21.15.94:0/672937174 reap_dead start 2024-04-25T15:00:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 shutdown_connections 2024-04-25T15:00:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 >> 172.21.15.94:0/672937174 conn(0x7f0eb4077fb0 msgr2=0x7f0eb40ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 shutdown_connections 2024-04-25T15:00:31.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:28.383+0000 7f0ebc400700 1 -- 172.21.15.94:0/672937174 wait complete. 2024-04-25T15:00:32.038 INFO:teuthology.orchestra.run.smithi094.stdout:201863462961 2024-04-25T15:00:32.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T15:00:32.669 INFO:teuthology.orchestra.run.smithi094.stdout:146028888124 2024-04-25T15:00:32.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T15:00:32.912 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:32 smithi094 ceph-mon[27862]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:32.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:32 smithi138 ceph-mon[32875]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:32.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:32 smithi137 ceph-mon[32439]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:33.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 -- 172.21.15.94:0/411714878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0107940 msgr2=0x7efef0107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 --2- 172.21.15.94:0/411714878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0107940 0x7efef0107d20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efee00099c0 tx=0x7efee0009cd0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 -- 172.21.15.94:0/411714878 shutdown_connections 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 --2- 172.21.15.94:0/411714878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef01034b0 0x7efef010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 --2- 172.21.15.94:0/411714878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0107940 0x7efef0107d20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 --2- 172.21.15.94:0/411714878 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef0102af0 0x7efef0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 -- 172.21.15.94:0/411714878 >> 172.21.15.94:0/411714878 conn(0x7efef0079ba0 msgr2=0x7efef0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 -- 172.21.15.94:0/411714878 shutdown_connections 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.008+0000 7efef7000700 1 -- 172.21.15.94:0/411714878 wait complete. 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.009+0000 7efef7000700 1 Processor -- start 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efef7000700 1 -- start start 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efef7000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0102af0 0x7efef0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:33.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efef7000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef01034b0 0x7efef0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efef7000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0107940 0x7efef00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efef7000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7efef0112c90 con 0x7efef0102af0 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efef7000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7efef0112b10 con 0x7efef01034b0 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efef7000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efef0112e10 con 0x7efef0107940 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efeefe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0102af0 0x7efef0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efeefe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0102af0 0x7efef0074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38722/0 (socket says 172.21.15.94:38722) 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.010+0000 7efeefe00700 1 -- 172.21.15.94:0/3194651606 learned_addr learned my addr 172.21.15.94:0/3194651606 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efeef400700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef01034b0 0x7efef0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efef5000700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0107940 0x7efef00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efeef400700 1 -- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0107940 msgr2=0x7efef00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efeef400700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0107940 0x7efef00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efeef400700 1 -- 172.21.15.94:0/3194651606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0102af0 msgr2=0x7efef0074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efeef400700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efef0102af0 0x7efef0074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efeef400700 1 -- 172.21.15.94:0/3194651606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef01a7650 con 0x7efef01034b0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efef5000700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efef0107940 0x7efef00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.011+0000 7efeef400700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef01034b0 0x7efef0075280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e9f0 tx=0x7efedc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.012+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc00c9c0 con 0x7efef01034b0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.012+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efedc00cb60 con 0x7efef01034b0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.012+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc018680 con 0x7efef01034b0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.013+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef01a7940 con 0x7efef01034b0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.013+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01a7e30 con 0x7efef01034b0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.013+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efef0063110 con 0x7efef01034b0 2024-04-25T15:00:33.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.017+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efedc010430 con 0x7efef01034b0 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.019+0000 7efeecc00700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efed40a3d50 0x7efed40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.019+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7efedc0d0b90 con 0x7efef01034b0 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.019+0000 7efeecc00700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] conn(0x7efed40ad910 0x7efed40afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.019+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 --> [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efedc016ec0 con 0x7efed40ad910 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.019+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7efedc0d0fc0 con 0x7efef01034b0 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.019+0000 7efeefe00700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efed40a3d50 0x7efed40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.020+0000 7efef5000700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] conn(0x7efed40ad910 0x7efed40afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.020+0000 7efef5000700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] conn(0x7efed40ad910 0x7efed40afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.020+0000 7efeefe00700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efed40a3d50 0x7efed40a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efee0003760 tx=0x7efee0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.021+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== osd.8 v2:172.21.15.138:6816/3378906429 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efedc016ec0 con 0x7efed40ad910 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.044+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 --> [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efef00635e0 con 0x7efed40ad910 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.044+0000 7efeecc00700 1 -- 172.21.15.94:0/3194651606 <== osd.8 v2:172.21.15.138:6816/3378906429 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efef00635e0 con 0x7efed40ad910 2024-04-25T15:00:33.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.045+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] conn(0x7efed40ad910 msgr2=0x7efed40afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.045+0000 7efef7000700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] conn(0x7efed40ad910 0x7efed40afdb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.045+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efed40a3d50 msgr2=0x7efed40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.045+0000 7efef7000700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7efed40a3d50 0x7efed40a6210 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efee0003760 tx=0x7efee0005fd0 comp rx=0 tx=0).stop 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.045+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef01034b0 msgr2=0x7efef0075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.045+0000 7efef7000700 1 --2- 172.21.15.94:0/3194651606 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7efef01034b0 0x7efef0075280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efedc00e9f0 tx=0x7efedc00ed00 comp rx=0 tx=0).stop 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.046+0000 7efef5000700 1 -- 172.21.15.94:0/3194651606 reap_dead start 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.046+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 shutdown_connections 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.046+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 >> 172.21.15.94:0/3194651606 conn(0x7efef0079ba0 msgr2=0x7efef0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.046+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 shutdown_connections 2024-04-25T15:00:33.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:33.046+0000 7efef7000700 1 -- 172.21.15.94:0/3194651606 wait complete. 2024-04-25T15:00:34.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.176+0000 7f9dd0800700 1 -- 172.21.15.94:0/132151996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 msgr2=0x7f9dc81052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.176+0000 7f9dd0800700 1 --2- 172.21.15.94:0/132151996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 0x7f9dc81052a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0099c0 tx=0x7f9dbc009cd0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.177+0000 7f9dd0800700 1 -- 172.21.15.94:0/132151996 shutdown_connections 2024-04-25T15:00:34.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.177+0000 7f9dd0800700 1 --2- 172.21.15.94:0/132151996 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dc8109fa0 0x7f9dc8111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.177+0000 7f9dd0800700 1 --2- 172.21.15.94:0/132151996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 0x7f9dc81052a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.177+0000 7f9dd0800700 1 --2- 172.21.15.94:0/132151996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc8105870 0x7f9dc81098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.177+0000 7f9dd0800700 1 -- 172.21.15.94:0/132151996 >> 172.21.15.94:0/132151996 conn(0x7f9dc8100750 msgr2=0x7f9dc8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.177+0000 7f9dd0800700 1 -- 172.21.15.94:0/132151996 shutdown_connections 2024-04-25T15:00:34.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.177+0000 7f9dd0800700 1 -- 172.21.15.94:0/132151996 wait complete. 2024-04-25T15:00:34.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 Processor -- start 2024-04-25T15:00:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 -- start start 2024-04-25T15:00:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 0x7f9dc819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dc8105870 0x7f9dc819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc8109fa0 0x7f9dc819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8114e80 con 0x7f9dc8104ec0 2024-04-25T15:00:34.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8114d00 con 0x7f9dc8105870 2024-04-25T15:00:34.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.178+0000 7f9dd0800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc8115000 con 0x7f9dc8109fa0 2024-04-25T15:00:34.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcde00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 0x7f9dc819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dce800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc8109fa0 0x7f9dc819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dce800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc8109fa0 0x7f9dc819fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:39870/0 (socket says 172.21.15.94:39870) 2024-04-25T15:00:34.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dce800700 1 -- 172.21.15.94:0/1537273697 learned_addr learned my addr 172.21.15.94:0/1537273697 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:34.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcd400700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dc8105870 0x7f9dc819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcde00700 1 -- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc8109fa0 msgr2=0x7f9dc819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcde00700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc8109fa0 0x7f9dc819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcde00700 1 -- 172.21.15.94:0/1537273697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dc8105870 msgr2=0x7f9dc819b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcde00700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9dc8105870 0x7f9dc819b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcde00700 1 -- 172.21.15.94:0/1537273697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc81a0360 con 0x7f9dc8104ec0 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dce800700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc8109fa0 0x7f9dc819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dcde00700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 0x7f9dc819b350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009f50 tx=0x7f9dbc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.179+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc027070 con 0x7f9dc8104ec0 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.180+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc003f20 con 0x7f9dc8104ec0 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.180+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc022bd0 con 0x7f9dc8104ec0 2024-04-25T15:00:34.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.180+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc81a05f0 con 0x7f9dc8104ec0 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.180+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc81a7ed0 con 0x7f9dc8104ec0 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.181+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9dc8063150 con 0x7f9dc8104ec0 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.183+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9dbc02a070 con 0x7f9dc8104ec0 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.184+0000 7f9dc2a00700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9db00a3d50 0x7f9db00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.184+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9dbc0d1a50 con 0x7f9dc8104ec0 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.184+0000 7f9dcd400700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9db00a3d50 0x7f9db00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.185+0000 7f9dc2a00700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] conn(0x7f9db00ad910 0x7f9db00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.185+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 --> [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9dbc010ea0 con 0x7f9db00ad910 2024-04-25T15:00:34.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.185+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9dbc0d1e00 con 0x7f9dc8104ec0 2024-04-25T15:00:34.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.185+0000 7f9dcd400700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9db00a3d50 0x7f9db00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9dc810b550 tx=0x7f9db800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:34.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.185+0000 7f9dce800700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] conn(0x7f9db00ad910 0x7f9db00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.186+0000 7f9dce800700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] conn(0x7f9db00ad910 0x7f9db00afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:34.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.187+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== osd.2 v2:172.21.15.138:6800/3663095065 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9dbc010ea0 con 0x7f9db00ad910 2024-04-25T15:00:34.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.204+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 --> [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9dc8063550 con 0x7f9db00ad910 2024-04-25T15:00:34.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.205+0000 7f9dc2a00700 1 -- 172.21.15.94:0/1537273697 <== osd.2 v2:172.21.15.138:6800/3663095065 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9dc8063550 con 0x7f9db00ad910 2024-04-25T15:00:34.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] conn(0x7f9db00ad910 msgr2=0x7f9db00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] conn(0x7f9db00ad910 0x7f9db00afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9db00a3d50 msgr2=0x7f9db00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9db00a3d50 0x7f9db00a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9dc810b550 tx=0x7f9db800b410 comp rx=0 tx=0).stop 2024-04-25T15:00:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 msgr2=0x7f9dc819b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 --2- 172.21.15.94:0/1537273697 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dc8104ec0 0x7f9dc819b350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009f50 tx=0x7f9dbc0047b0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dce800700 1 -- 172.21.15.94:0/1537273697 reap_dead start 2024-04-25T15:00:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 shutdown_connections 2024-04-25T15:00:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 >> 172.21.15.94:0/1537273697 conn(0x7f9dc8100750 msgr2=0x7f9dc8102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 shutdown_connections 2024-04-25T15:00:34.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.206+0000 7f9dd0800700 1 -- 172.21.15.94:0/1537273697 wait complete. 2024-04-25T15:00:34.661 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:34 smithi094 ceph-mon[27862]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:34.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 -- 172.21.15.94:0/154978733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18113980 msgr2=0x7fce18115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 --2- 172.21.15.94:0/154978733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18113980 0x7fce18115d70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fce10009990 tx=0x7fce10009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 -- 172.21.15.94:0/154978733 shutdown_connections 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 --2- 172.21.15.94:0/154978733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18113980 0x7fce18115d70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 --2- 172.21.15.94:0/154978733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce18074de0 0x7fce180751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 --2- 172.21.15.94:0/154978733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce18075700 0x7fce18113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 -- 172.21.15.94:0/154978733 >> 172.21.15.94:0/154978733 conn(0x7fce180fe4c0 msgr2=0x7fce181008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 -- 172.21.15.94:0/154978733 shutdown_connections 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.406+0000 7fce17200700 1 -- 172.21.15.94:0/154978733 wait complete. 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.407+0000 7fce17200700 1 Processor -- start 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce17200700 1 -- start start 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce17200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18074de0 0x7fce1819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce17200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce18075700 0x7fce1819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce17200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce18113980 0x7fce181a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce17200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fce18119170 con 0x7fce18074de0 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce17200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fce18118ff0 con 0x7fce18075700 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce17200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce181192f0 con 0x7fce18113980 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.408+0000 7fce15e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18074de0 0x7fce1819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18074de0 0x7fce1819f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38752/0 (socket says 172.21.15.94:38752) 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 -- 172.21.15.94:0/2079510861 learned_addr learned my addr 172.21.15.94:0/2079510861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce16800700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce18113980 0x7fce181a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15400700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce18075700 0x7fce1819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 -- 172.21.15.94:0/2079510861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce18113980 msgr2=0x7fce181a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce18113980 0x7fce181a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 -- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce18075700 msgr2=0x7fce1819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce18075700 0x7fce1819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 -- 172.21.15.94:0/2079510861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce181a4650 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15400700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fce18075700 0x7fce1819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce15e00700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18074de0 0x7fce1819f640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce0800ef10 tx=0x7fce0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.409+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0800ca70 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.410+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce08019070 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.410+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce181a48e0 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.410+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08014480 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.410+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce181ac1c0 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.410+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce180631d0 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.413+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fce08012050 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.414+0000 7fce0ea00700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcdfc0a3d50 0x7fcdfc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.414+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fce080d1060 con 0x7fce18074de0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.414+0000 7fce15400700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcdfc0a3d50 0x7fcdfc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.414+0000 7fce0ea00700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] conn(0x7fcdfc0ad910 0x7fcdfc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:34.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.414+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 --> [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce08008ec0 con 0x7fcdfc0ad910 2024-04-25T15:00:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.414+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fce080d1490 con 0x7fce18074de0 2024-04-25T15:00:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.415+0000 7fce16800700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] conn(0x7fcdfc0ad910 0x7fcdfc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.415+0000 7fce15400700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcdfc0a3d50 0x7fcdfc0a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fce18106770 tx=0x7fce04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.415+0000 7fce16800700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] conn(0x7fcdfc0ad910 0x7fcdfc0afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.416+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== osd.10 v2:172.21.15.137:6824/3255175900 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce08008ec0 con 0x7fcdfc0ad910 2024-04-25T15:00:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.434+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 --> [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce180635d0 con 0x7fcdfc0ad910 2024-04-25T15:00:34.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.434+0000 7fce0ea00700 1 -- 172.21.15.94:0/2079510861 <== osd.10 v2:172.21.15.137:6824/3255175900 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fce180635d0 con 0x7fcdfc0ad910 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] conn(0x7fcdfc0ad910 msgr2=0x7fcdfc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] conn(0x7fcdfc0ad910 0x7fcdfc0afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcdfc0a3d50 msgr2=0x7fcdfc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcdfc0a3d50 0x7fcdfc0a6210 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fce18106770 tx=0x7fce04009260 comp rx=0 tx=0).stop 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18074de0 msgr2=0x7fce1819f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 --2- 172.21.15.94:0/2079510861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce18074de0 0x7fce1819f640 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fce0800ef10 tx=0x7fce0800c3a0 comp rx=0 tx=0).stop 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce16800700 1 -- 172.21.15.94:0/2079510861 reap_dead start 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 shutdown_connections 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 >> 172.21.15.94:0/2079510861 conn(0x7fce180fe4c0 msgr2=0x7fce180ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 shutdown_connections 2024-04-25T15:00:34.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.435+0000 7fce17200700 1 -- 172.21.15.94:0/2079510861 wait complete. 2024-04-25T15:00:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:34 smithi138 ceph-mon[32875]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:34 smithi137 ceph-mon[32439]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.425+0000 7f7619c00700 1 -- 172.21.15.94:0/1783760741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761406a750 msgr2=0x7f761406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.425+0000 7f7619c00700 1 --2- 172.21.15.94:0/1783760741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761406a750 0x7f761406abb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7600009990 tx=0x7f7600009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.425+0000 7f7619c00700 1 -- 172.21.15.94:0/1783760741 shutdown_connections 2024-04-25T15:00:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.425+0000 7f7619c00700 1 --2- 172.21.15.94:0/1783760741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761406b0f0 0x7f761410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.425+0000 7f7619c00700 1 --2- 172.21.15.94:0/1783760741 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7614108a30 0x7f7614108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.425+0000 7f7619c00700 1 --2- 172.21.15.94:0/1783760741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f761406a750 0x7f761406abb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.425+0000 7f7619c00700 1 -- 172.21.15.94:0/1783760741 >> 172.21.15.94:0/1783760741 conn(0x7f76140788f0 msgr2=0x7f7614078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.426+0000 7f7619c00700 1 -- 172.21.15.94:0/1783760741 shutdown_connections 2024-04-25T15:00:35.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.426+0000 7f7619c00700 1 -- 172.21.15.94:0/1783760741 wait complete. 2024-04-25T15:00:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 Processor -- start 2024-04-25T15:00:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 -- start start 2024-04-25T15:00:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761406a750 0x7f761419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761406b0f0 0x7f761419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614108a30 0x7f761419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7614112bd0 con 0x7f761406b0f0 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7614112a50 con 0x7f761406a750 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7619c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7614112d50 con 0x7f7614108a30 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7612000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761406b0f0 0x7f761419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7612000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761406b0f0 0x7f761419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38790/0 (socket says 172.21.15.94:38790) 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.427+0000 7f7612000700 1 -- 172.21.15.94:0/189142594 learned_addr learned my addr 172.21.15.94:0/189142594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612a00700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761406a750 0x7f761419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7613400700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614108a30 0x7f761419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612a00700 1 -- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614108a30 msgr2=0x7f761419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612a00700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614108a30 0x7f761419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612a00700 1 -- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761406b0f0 msgr2=0x7f761419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612a00700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761406b0f0 0x7f761419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612a00700 1 -- 172.21.15.94:0/189142594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76141a0370 con 0x7f761406a750 2024-04-25T15:00:35.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612000700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f761406b0f0 0x7f761419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.428+0000 7f7612a00700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761406a750 0x7f761419b3b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f760400e9f0 tx=0x7f760400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.429+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604004220 con 0x7f761406a750 2024-04-25T15:00:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.429+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76141a0660 con 0x7f761406a750 2024-04-25T15:00:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.429+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76040043c0 con 0x7f761406a750 2024-04-25T15:00:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.429+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604005020 con 0x7f761406a750 2024-04-25T15:00:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.429+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76141a7ee0 con 0x7f761406a750 2024-04-25T15:00:35.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.430+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f76141029d0 con 0x7f761406a750 2024-04-25T15:00:35.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.432+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7604004750 con 0x7f761406a750 2024-04-25T15:00:35.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.433+0000 7f760b400700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f75f80a3cd0 0x7f75f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.433+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f76040d06e0 con 0x7f761406a750 2024-04-25T15:00:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.434+0000 7f7612000700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f75f80a3cd0 0x7f75f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.434+0000 7f760b400700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] conn(0x7f75f80ad890 0x7f75f80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.434+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 --> [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7604016ec0 con 0x7f75f80ad890 2024-04-25T15:00:35.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.434+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f76040d0b10 con 0x7f761406a750 2024-04-25T15:00:35.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.434+0000 7f7613400700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] conn(0x7f75f80ad890 0x7f75f80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.434+0000 7f7612000700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f75f80a3cd0 0x7f75f80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7600009750 tx=0x7f760001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:35.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.435+0000 7f7613400700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] conn(0x7f75f80ad890 0x7f75f80afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:35.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.436+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== osd.5 v2:172.21.15.138:6808/100945192 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7604016ec0 con 0x7f75f80ad890 2024-04-25T15:00:35.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.454+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 --> [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7614104170 con 0x7f75f80ad890 2024-04-25T15:00:35.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.454+0000 7f760b400700 1 -- 172.21.15.94:0/189142594 <== osd.5 v2:172.21.15.138:6808/100945192 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7614104170 con 0x7f75f80ad890 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] conn(0x7f75f80ad890 msgr2=0x7f75f80afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] conn(0x7f75f80ad890 0x7f75f80afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f75f80a3cd0 msgr2=0x7f75f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f75f80a3cd0 0x7f75f80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7600009750 tx=0x7f760001a040 comp rx=0 tx=0).stop 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761406a750 msgr2=0x7f761419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 --2- 172.21.15.94:0/189142594 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f761406a750 0x7f761419b3b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f760400e9f0 tx=0x7f760400ed00 comp rx=0 tx=0).stop 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7613400700 1 -- 172.21.15.94:0/189142594 reap_dead start 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 shutdown_connections 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.455+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 >> 172.21.15.94:0/189142594 conn(0x7f76140788f0 msgr2=0x7f76140ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.456+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 shutdown_connections 2024-04-25T15:00:35.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:35.456+0000 7f7619c00700 1 -- 172.21.15.94:0/189142594 wait complete. 2024-04-25T15:00:35.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.819+0000 7fa276e00700 1 -- 172.21.15.94:0/2637089030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270103310 msgr2=0x7fa27010f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.819+0000 7fa276e00700 1 --2- 172.21.15.94:0/2637089030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270103310 0x7fa27010f820 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b0a0 tx=0x7fa26800b3b0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.820+0000 7fa276e00700 1 -- 172.21.15.94:0/2637089030 shutdown_connections 2024-04-25T15:00:35.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.820+0000 7fa276e00700 1 --2- 172.21.15.94:0/2637089030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270103310 0x7fa27010f820 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.820+0000 7fa276e00700 1 --2- 172.21.15.94:0/2637089030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270108950 0x7fa270108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.820+0000 7fa276e00700 1 --2- 172.21.15.94:0/2637089030 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa270102950 0x7fa270102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.820+0000 7fa276e00700 1 -- 172.21.15.94:0/2637089030 >> 172.21.15.94:0/2637089030 conn(0x7fa2700fe460 msgr2=0x7fa270100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.820+0000 7fa276e00700 1 -- 172.21.15.94:0/2637089030 shutdown_connections 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.820+0000 7fa276e00700 1 -- 172.21.15.94:0/2637089030 wait complete. 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 Processor -- start 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 -- start start 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270102950 0x7fa27019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103310 0x7fa27019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa270108950 0x7fa27019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa270112bb0 con 0x7fa270102950 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa270112a30 con 0x7fa270108950 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.821+0000 7fa276e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa270112d30 con 0x7fa270103310 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270102950 0x7fa27019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270102950 0x7fa27019b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38764/0 (socket says 172.21.15.94:38764) 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 -- 172.21.15.94:0/3514369268 learned_addr learned my addr 172.21.15.94:0/3514369268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275000700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103310 0x7fa27019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa276400700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa270108950 0x7fa27019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 -- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103310 msgr2=0x7fa27019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa270103310 0x7fa27019b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 -- 172.21.15.94:0/3514369268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa270108950 msgr2=0x7fa27019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa270108950 0x7fa27019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 -- 172.21.15.94:0/3514369268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2701a0270 con 0x7fa270102950 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa276400700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa270108950 0x7fa27019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa275a00700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270102950 0x7fa27019b260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9c0 tx=0x7fa26400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.822+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26400cbd0 con 0x7fa270102950 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.823+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2701a0560 con 0x7fa270102950 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.823+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa26400cd70 con 0x7fa270102950 2024-04-25T15:00:35.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.823+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa264005610 con 0x7fa270102950 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.823+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701a7de0 con 0x7fa270102950 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.823+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa27010f7f0 con 0x7fa270102950 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.826+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa264004750 con 0x7fa270102950 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.827+0000 7fa26e000700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa2580a3cd0 0x7fa2580a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.827+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fa2640d0ce0 con 0x7fa270102950 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.827+0000 7fa275000700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa2580a3cd0 0x7fa2580a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.827+0000 7fa26e000700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] conn(0x7fa2580ad890 0x7fa2580afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.827+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 --> [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa26400be50 con 0x7fa2580ad890 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.827+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fa2640d1110 con 0x7fa270102950 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.828+0000 7fa276400700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] conn(0x7fa2580ad890 0x7fa2580afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.828+0000 7fa275000700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa2580a3cd0 0x7fa2580a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2701082f0 tx=0x7fa25c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.828+0000 7fa276400700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] conn(0x7fa2580ad890 0x7fa2580afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:35.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.829+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== osd.11 v2:172.21.15.138:6824/883275794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa26400be50 con 0x7fa2580ad890 2024-04-25T15:00:35.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.846+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 --> [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa270104090 con 0x7fa2580ad890 2024-04-25T15:00:35.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa26e000700 1 -- 172.21.15.94:0/3514369268 <== osd.11 v2:172.21.15.138:6824/883275794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa270104090 con 0x7fa2580ad890 2024-04-25T15:00:35.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] conn(0x7fa2580ad890 msgr2=0x7fa2580afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa276e00700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] conn(0x7fa2580ad890 0x7fa2580afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa2580a3cd0 msgr2=0x7fa2580a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa276e00700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fa2580a3cd0 0x7fa2580a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa2701082f0 tx=0x7fa25c009190 comp rx=0 tx=0).stop 2024-04-25T15:00:35.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270102950 msgr2=0x7fa27019b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:35.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa276e00700 1 --2- 172.21.15.94:0/3514369268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa270102950 0x7fa27019b260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa26400e9c0 tx=0x7fa26400ecd0 comp rx=0 tx=0).stop 2024-04-25T15:00:35.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.847+0000 7fa276400700 1 -- 172.21.15.94:0/3514369268 reap_dead start 2024-04-25T15:00:35.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.848+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 shutdown_connections 2024-04-25T15:00:35.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.848+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 >> 172.21.15.94:0/3514369268 conn(0x7fa2700fe460 msgr2=0x7fa27010e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:35.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.848+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 shutdown_connections 2024-04-25T15:00:35.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:34.848+0000 7fa276e00700 1 -- 172.21.15.94:0/3514369268 wait complete. 2024-04-25T15:00:36.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:36 smithi094 ceph-mon[27862]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:00:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:36 smithi138 ceph-mon[32875]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:00:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:36 smithi137 ceph-mon[32439]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:00:37.835 INFO:teuthology.orchestra.run.smithi094.stdout:313532612635 2024-04-25T15:00:37.836 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T15:00:38.551 INFO:teuthology.orchestra.run.smithi094.stdout:257698037799 2024-04-25T15:00:38.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T15:00:38.740 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:38 smithi094 ceph-mon[27862]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:38 smithi138 ceph-mon[32875]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:38 smithi137 ceph-mon[32439]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:39.231 INFO:teuthology.orchestra.run.smithi094.stdout:347892350998 2024-04-25T15:00:39.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T15:00:39.470 INFO:teuthology.orchestra.run.smithi094.stdout:77309411401 2024-04-25T15:00:39.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T15:00:39.700 INFO:teuthology.orchestra.run.smithi094.stdout:167503724599 2024-04-25T15:00:39.701 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 -- 172.21.15.94:0/259150844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70075590 msgr2=0x7fcb701136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 --2- 172.21.15.94:0/259150844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70075590 0x7fcb701136d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009990 tx=0x7fcb5c009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 -- 172.21.15.94:0/259150844 shutdown_connections 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 --2- 172.21.15.94:0/259150844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb70113c10 0x7fcb70116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 --2- 172.21.15.94:0/259150844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70074c70 0x7fcb70075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 --2- 172.21.15.94:0/259150844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70075590 0x7fcb701136d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 -- 172.21.15.94:0/259150844 >> 172.21.15.94:0/259150844 conn(0x7fcb70079bf0 msgr2=0x7fcb7007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 -- 172.21.15.94:0/259150844 shutdown_connections 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.694+0000 7fcb78600700 1 -- 172.21.15.94:0/259150844 wait complete. 2024-04-25T15:00:39.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.695+0000 7fcb78600700 1 Processor -- start 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.696+0000 7fcb78600700 1 -- start start 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.696+0000 7fcb78600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70074c70 0x7fcb7019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.696+0000 7fcb78600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb70075590 0x7fcb7019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.696+0000 7fcb78600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70113c10 0x7fcb701a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.696+0000 7fcb78600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70119200 con 0x7fcb70113c10 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.696+0000 7fcb78600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70119080 con 0x7fcb70074c70 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.696+0000 7fcb78600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb70119380 con 0x7fcb70075590 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70074c70 0x7fcb7019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70074c70 0x7fcb7019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58520/0 (socket says 172.21.15.94:58520) 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 -- 172.21.15.94:0/355145600 learned_addr learned my addr 172.21.15.94:0/355145600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75200700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb70075590 0x7fcb7019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb76600700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70113c10 0x7fcb701a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 -- 172.21.15.94:0/355145600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb70075590 msgr2=0x7fcb7019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb70075590 0x7fcb7019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 -- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70113c10 msgr2=0x7fcb701a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcb70113c10 0x7fcb701a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 -- 172.21.15.94:0/355145600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb701a4700 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.697+0000 7fcb75c00700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70074c70 0x7fcb7019f6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e9a0 tx=0x7fcb6400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.698+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6400cbd0 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.698+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb6400cd70 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.698+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64018680 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.698+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb701a49f0 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.698+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb701ac270 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.699+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb70110110 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.702+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb64010430 con 0x7fcb70074c70 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.703+0000 7fcb6e000700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcb580a3d50 0x7fcb580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.704+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fcb640d1260 con 0x7fcb70074c70 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.704+0000 7fcb75200700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcb580a3d50 0x7fcb580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.704+0000 7fcb6e000700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] conn(0x7fcb580ad910 0x7fcb580afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.704+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 --> [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcb640056b0 con 0x7fcb580ad910 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.704+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fcb640d1610 con 0x7fcb70074c70 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.704+0000 7fcb76600700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] conn(0x7fcb580ad910 0x7fcb580afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.705+0000 7fcb75200700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcb580a3d50 0x7fcb580a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00b2e0 tx=0x7fcb5c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.705+0000 7fcb76600700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] conn(0x7fcb580ad910 0x7fcb580afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.707+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== osd.9 v2:172.21.15.94:6826/1308519263 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcb640056b0 con 0x7fcb580ad910 2024-04-25T15:00:39.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.725+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 --> [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcb70076390 con 0x7fcb580ad910 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.726+0000 7fcb6e000700 1 -- 172.21.15.94:0/355145600 <== osd.9 v2:172.21.15.94:6826/1308519263 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fcb70076390 con 0x7fcb580ad910 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.726+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] conn(0x7fcb580ad910 msgr2=0x7fcb580afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.726+0000 7fcb78600700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] conn(0x7fcb580ad910 0x7fcb580afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.726+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcb580a3d50 msgr2=0x7fcb580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.726+0000 7fcb78600700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fcb580a3d50 0x7fcb580a6210 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00b2e0 tx=0x7fcb5c005960 comp rx=0 tx=0).stop 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.727+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70074c70 msgr2=0x7fcb7019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.727+0000 7fcb78600700 1 --2- 172.21.15.94:0/355145600 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcb70074c70 0x7fcb7019f6f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400e9a0 tx=0x7fcb6400ecb0 comp rx=0 tx=0).stop 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.727+0000 7fcb76600700 1 -- 172.21.15.94:0/355145600 reap_dead start 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.727+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 shutdown_connections 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.727+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 >> 172.21.15.94:0/355145600 conn(0x7fcb70079bf0 msgr2=0x7fcb7010f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.727+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 shutdown_connections 2024-04-25T15:00:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:39.727+0000 7fcb78600700 1 -- 172.21.15.94:0/355145600 wait complete. 2024-04-25T15:00:40.649 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:40 smithi094 ceph-mon[27862]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:40.653 INFO:teuthology.orchestra.run.smithi094.stdout:292057776162 2024-04-25T15:00:40.653 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T15:00:40.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 -- 172.21.15.94:0/4208861000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde01092d0 msgr2=0x7fdde01096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:40.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 --2- 172.21.15.94:0/4208861000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde01092d0 0x7fdde01096b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fddd00099c0 tx=0x7fddd0009cd0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 -- 172.21.15.94:0/4208861000 shutdown_connections 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 --2- 172.21.15.94:0/4208861000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde010e3b0 0x7fdde0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 --2- 172.21.15.94:0/4208861000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde01092d0 0x7fdde01096b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 --2- 172.21.15.94:0/4208861000 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde0109c80 0x7fdde010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 -- 172.21.15.94:0/4208861000 >> 172.21.15.94:0/4208861000 conn(0x7fdde0076210 msgr2=0x7fdde0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 -- 172.21.15.94:0/4208861000 shutdown_connections 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.342+0000 7fdde5600700 1 -- 172.21.15.94:0/4208861000 wait complete. 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 Processor -- start 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 -- start start 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde01092d0 0x7fdde019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0109c80 0x7fdde019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde010e3b0 0x7fdde01a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fdde01192c0 con 0x7fdde010e3b0 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0119140 con 0x7fdde01092d0 2024-04-25T15:00:40.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.343+0000 7fdde5600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde0119440 con 0x7fdde0109c80 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fddde000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0109c80 0x7fdde019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde010e3b0 0x7fdde01a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddea00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde01092d0 0x7fdde019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fddde000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0109c80 0x7fdde019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:56612/0 (socket says 172.21.15.94:56612) 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fddde000700 1 -- 172.21.15.94:0/2054472814 learned_addr learned my addr 172.21.15.94:0/2054472814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde010e3b0 0x7fdde01a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44992/0 (socket says 172.21.15.94:44992) 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 -- 172.21.15.94:0/2054472814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0109c80 msgr2=0x7fdde019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0109c80 0x7fdde019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 -- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde01092d0 msgr2=0x7fdde019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde01092d0 0x7fdde019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 -- 172.21.15.94:0/2054472814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde01a4750 con 0x7fdde010e3b0 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddea00700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fdde01092d0 0x7fdde019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.344+0000 7fdddf400700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde010e3b0 0x7fdde01a4010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddd800efc0 tx=0x7fddd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.345+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd80092e0 con 0x7fdde010e3b0 2024-04-25T15:00:40.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.345+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddd8004bb0 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.345+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8004020 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.345+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde01a49e0 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.345+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde01ac2c0 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.346+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdde0063190 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.348+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fddd801b030 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.349+0000 7fddd7400700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fddc40a3cd0 0x7fddc40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.349+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fddd80d09e0 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.349+0000 7fdddea00700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fddc40a3cd0 0x7fddc40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.349+0000 7fddd7400700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] conn(0x7fddc40ad890 0x7fddc40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.349+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 --> [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fddd800fe50 con 0x7fddc40ad890 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.349+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fddd80d0d90 con 0x7fdde010e3b0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.350+0000 7fddde000700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] conn(0x7fddc40ad890 0x7fddc40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.350+0000 7fdddea00700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fddc40a3cd0 0x7fddc40a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fddd0006010 tx=0x7fddd0005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:40.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.350+0000 7fddde000700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] conn(0x7fddc40ad890 0x7fddc40afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.351+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== osd.7 v2:172.21.15.137:6816/2535537173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fddd800fe50 con 0x7fddc40ad890 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.369+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 --> [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdde0063610 con 0x7fddc40ad890 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fddd7400700 1 -- 172.21.15.94:0/2054472814 <== osd.7 v2:172.21.15.137:6816/2535537173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdde0063610 con 0x7fddc40ad890 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] conn(0x7fddc40ad890 msgr2=0x7fddc40afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] conn(0x7fddc40ad890 0x7fddc40afd30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fddc40a3cd0 msgr2=0x7fddc40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fddc40a3cd0 0x7fddc40a6190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fddd0006010 tx=0x7fddd0005ab0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde010e3b0 msgr2=0x7fdde01a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:40.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 --2- 172.21.15.94:0/2054472814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdde010e3b0 0x7fdde01a4010 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fddd800efc0 tx=0x7fddd800c3a0 comp rx=0 tx=0).stop 2024-04-25T15:00:40.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdddf400700 1 -- 172.21.15.94:0/2054472814 reap_dead start 2024-04-25T15:00:40.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 shutdown_connections 2024-04-25T15:00:40.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.370+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 >> 172.21.15.94:0/2054472814 conn(0x7fdde0076210 msgr2=0x7fdde0077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:40.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.371+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 shutdown_connections 2024-04-25T15:00:40.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:40.371+0000 7fdde5600700 1 -- 172.21.15.94:0/2054472814 wait complete. 2024-04-25T15:00:40.940 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:40.949 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:40.955 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:40.955 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:40.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:40 smithi138 ceph-mon[32875]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:40.958 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:40 smithi137 ceph-mon[32439]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:41.477 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:41.484 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:41.484 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:41.512 INFO:teuthology.orchestra.run.smithi094.stdout:223338299437 2024-04-25T15:00:41.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T15:00:41.772 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:42.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:42 smithi094 ceph-mon[27862]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:42 smithi138 ceph-mon[32875]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:42 smithi137 ceph-mon[32439]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:44.649 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:44 smithi094 ceph-mon[27862]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:44 smithi138 ceph-mon[32875]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:44.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:44 smithi137 ceph-mon[32439]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:46.911 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:46 smithi094 ceph-mon[27862]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:46.912 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:46 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:46 smithi138 ceph-mon[32875]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:46 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:46 smithi137 ceph-mon[32439]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:46 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:00:48.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:48 smithi094 ceph-mon[27862]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:00:48.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:48 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:48 smithi138 ceph-mon[32875]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:00:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:48 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:48 smithi137 ceph-mon[32439]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:00:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:48.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:48 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:00:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 -- 172.21.15.94:0/753564267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 msgr2=0x7f06a8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 --2- 172.21.15.94:0/753564267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 0x7f06a8108e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0698009990 tx=0x7f0698009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 -- 172.21.15.94:0/753564267 shutdown_connections 2024-04-25T15:00:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 --2- 172.21.15.94:0/753564267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8103450 0x7f06a810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 --2- 172.21.15.94:0/753564267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 0x7f06a8108e30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 --2- 172.21.15.94:0/753564267 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06a8102a90 0x7f06a8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 -- 172.21.15.94:0/753564267 >> 172.21.15.94:0/753564267 conn(0x7f06a8079ba0 msgr2=0x7f06a8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 -- 172.21.15.94:0/753564267 shutdown_connections 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.729+0000 7f06acc00700 1 -- 172.21.15.94:0/753564267 wait complete. 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.730+0000 7f06acc00700 1 Processor -- start 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.730+0000 7f06acc00700 1 -- start start 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.730+0000 7f06acc00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8102a90 0x7f06a819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.730+0000 7f06acc00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06a8103450 0x7f06a819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.730+0000 7f06acc00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 0x7f06a819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06acc00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8112c60 con 0x7f06a8108a50 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06acc00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8112ae0 con 0x7f06a8103450 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06acc00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06a8112de0 con 0x7f06a8102a90 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8102a90 0x7f06a819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a6800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 0x7f06a819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8102a90 0x7f06a819b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40868/0 (socket says 172.21.15.94:40868) 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 -- 172.21.15.94:0/392397955 learned_addr learned my addr 172.21.15.94:0/392397955 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:49.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5400700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06a8103450 0x7f06a819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 -- 172.21.15.94:0/392397955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06a8103450 msgr2=0x7f06a819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06a8103450 0x7f06a819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.992 INFO:teuthology.orchestra.run.smithi094.stdout:146028888129 2024-04-25T15:00:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 -- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 msgr2=0x7f06a819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 0x7f06a819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.731+0000 7f06a5e00700 1 -- 172.21.15.94:0/392397955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a81a03a0 con 0x7f06a8102a90 2024-04-25T15:00:49.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.732+0000 7f06a5e00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8102a90 0x7f06a819b390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0698005910 tx=0x7f069800b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.732+0000 7f069ea00700 1 -- 172.21.15.94:0/392397955 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698027070 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.732+0000 7f069ea00700 1 -- 172.21.15.94:0/392397955 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0698022470 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.732+0000 7f069ea00700 1 -- 172.21.15.94:0/392397955 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698017650 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.732+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a81a0630 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.732+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a81a7f10 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.733+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a8104180 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.735+0000 7f069ea00700 1 -- 172.21.15.94:0/392397955 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f069802a070 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.736+0000 7f069ea00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f068c0a3d50 0x7f068c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.737+0000 7f069ea00700 1 -- 172.21.15.94:0/392397955 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f06980d1220 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.737+0000 7f06a5400700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f068c0a3d50 0x7f068c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.737+0000 7f06a5400700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f068c0a3d50 0x7f068c0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06a807a4a0 tx=0x7f0694009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.741+0000 7f069ea00700 1 -- 172.21.15.94:0/392397955 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0698013070 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.975+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f06a802df80 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.976+0000 7f069ea00700 1 -- 172.21.15.94:0/392397955 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f069809daf0 con 0x7f06a8102a90 2024-04-25T15:00:49.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f068c0a3d50 msgr2=0x7f068c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f068c0a3d50 0x7f068c0a6210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f06a807a4a0 tx=0x7f0694009260 comp rx=0 tx=0).stop 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8102a90 msgr2=0x7f06a819b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8102a90 0x7f06a819b390 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0698005910 tx=0x7f069800b670 comp rx=0 tx=0).stop 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 shutdown_connections 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f068c0a3d50 0x7f068c0a6210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06a8108a50 0x7f06a819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06a8102a90 0x7f06a819b390 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 --2- 172.21.15.94:0/392397955 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f06a8103450 0x7f06a819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 >> 172.21.15.94:0/392397955 conn(0x7f06a8079ba0 msgr2=0x7f06a810df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 shutdown_connections 2024-04-25T15:00:49.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:49.979+0000 7f06acc00700 1 -- 172.21.15.94:0/392397955 wait complete. 2024-04-25T15:00:50.527 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:50.903 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:50 smithi094 ceph-mon[27862]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:50.904 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:50 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/392397955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:00:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:50 smithi138 ceph-mon[32875]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:50 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/392397955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:00:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:50 smithi137 ceph-mon[32439]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:50 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/392397955' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:00:52.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:52 smithi094 ceph-mon[27862]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:52.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:52 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/488677025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:00:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 -- 172.21.15.94:0/2507603189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 msgr2=0x7f3a5c115cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 --2- 172.21.15.94:0/2507603189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 0x7f3a5c115cd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3a54009f90 tx=0x7f3a5400b330 comp rx=0 tx=0).stop 2024-04-25T15:00:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 -- 172.21.15.94:0/2507603189 shutdown_connections 2024-04-25T15:00:52.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 --2- 172.21.15.94:0/2507603189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 0x7f3a5c115cd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 --2- 172.21.15.94:0/2507603189 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c076f70 0x7f3a5c077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 --2- 172.21.15.94:0/2507603189 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a5c077890 0x7f3a5c1133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 -- 172.21.15.94:0/2507603189 >> 172.21.15.94:0/2507603189 conn(0x7f3a5c100750 msgr2=0x7f3a5c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 -- 172.21.15.94:0/2507603189 shutdown_connections 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.154+0000 7f3a61800700 1 -- 172.21.15.94:0/2507603189 wait complete. 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.155+0000 7f3a61800700 1 Processor -- start 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a61800700 1 -- start start 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a61800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a5c076f70 0x7f3a5c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a61800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c077890 0x7f3a5c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a61800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 0x7f3a5c1a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a61800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c119230 con 0x7f3a5c076f70 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a61800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c1190b0 con 0x7f3a5c077890 2024-04-25T15:00:52.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a61800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5c1193b0 con 0x7f3a5c1138e0 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a5aa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a5c076f70 0x7f3a5c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a53e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c077890 0x7f3a5c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a5aa00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a5c076f70 0x7f3a5c19f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35830/0 (socket says 172.21.15.94:35830) 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a5aa00700 1 -- 172.21.15.94:0/488677025 learned_addr learned my addr 172.21.15.94:0/488677025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a53e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c077890 0x7f3a5c19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46104/0 (socket says 172.21.15.94:46104) 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.156+0000 7f3a5b400700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 0x7f3a5c1a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53e00700 1 -- 172.21.15.94:0/488677025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 msgr2=0x7f3a5c1a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53e00700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 0x7f3a5c1a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53e00700 1 -- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a5c076f70 msgr2=0x7f3a5c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53e00700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a5c076f70 0x7f3a5c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53e00700 1 -- 172.21.15.94:0/488677025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5c1a46d0 con 0x7f3a5c077890 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53e00700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c077890 0x7f3a5c19fc00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400e9c0 tx=0x7f3a4400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:52.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53400700 1 -- 172.21.15.94:0/488677025 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400c9c0 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53400700 1 -- 172.21.15.94:0/488677025 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a4400cb60 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5c1a49c0 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a53400700 1 -- 172.21.15.94:0/488677025 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44018680 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.157+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a5c1ac240 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.158+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a5c069730 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.160+0000 7f3a53400700 1 -- 172.21.15.94:0/488677025 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3a44010430 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.162+0000 7f3a53400700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3a400a3d50 0x7f3a400a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.162+0000 7f3a53400700 1 -- 172.21.15.94:0/488677025 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f3a440d09f0 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.162+0000 7f3a5aa00700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3a400a3d50 0x7f3a400a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.162+0000 7f3a5aa00700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3a400a3d50 0x7f3a400a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009d60 tx=0x7f3a4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.164+0000 7f3a53400700 1 -- 172.21.15.94:0/488677025 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a4409d230 con 0x7f3a5c077890 2024-04-25T15:00:52.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.396+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3a5c02df80 con 0x7f3a5c077890 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stdout:34359738454 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.397+0000 7f3a53400700 1 -- 172.21.15.94:0/488677025 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a4409cb30 con 0x7f3a5c077890 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3a400a3d50 msgr2=0x7f3a400a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3a400a3d50 0x7f3a400a6210 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009d60 tx=0x7f3a4c009260 comp rx=0 tx=0).stop 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c077890 msgr2=0x7f3a5c19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c077890 0x7f3a5c19fc00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400e9c0 tx=0x7f3a4400ecd0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 shutdown_connections 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f3a400a3d50 0x7f3a400a6210 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a5c1138e0 0x7f3a5c1a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a5c076f70 0x7f3a5c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 --2- 172.21.15.94:0/488677025 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a5c077890 0x7f3a5c19fc00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 >> 172.21.15.94:0/488677025 conn(0x7f3a5c100750 msgr2=0x7f3a5c114550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 shutdown_connections 2024-04-25T15:00:52.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.400+0000 7f3a61800700 1 -- 172.21.15.94:0/488677025 wait complete. 2024-04-25T15:00:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:52 smithi138 ceph-mon[32875]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:52 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/488677025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:00:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:52 smithi137 ceph-mon[32439]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:52 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/488677025' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.579+0000 7fc7e4400700 1 -- 172.21.15.94:0/3400129957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 msgr2=0x7fc7dc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.579+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3400129957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc108e30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009990 tx=0x7fc7d0009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.580+0000 7fc7e4400700 1 -- 172.21.15.94:0/3400129957 shutdown_connections 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.580+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3400129957 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7dc103450 0x7fc7dc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.580+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3400129957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc108e30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.580+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3400129957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc102a90 0x7fc7dc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.580+0000 7fc7e4400700 1 -- 172.21.15.94:0/3400129957 >> 172.21.15.94:0/3400129957 conn(0x7fc7dc079ba0 msgr2=0x7fc7dc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.580+0000 7fc7e4400700 1 -- 172.21.15.94:0/3400129957 shutdown_connections 2024-04-25T15:00:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.580+0000 7fc7e4400700 1 -- 172.21.15.94:0/3400129957 wait complete. 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.581+0000 7fc7e4400700 1 Processor -- start 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.581+0000 7fc7e4400700 1 -- start start 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.581+0000 7fc7e4400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc102a90 0x7fc7dc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e4400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7dc103450 0x7fc7dc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e4400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e4400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc112c00 con 0x7fc7dc108a50 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e4400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc112a80 con 0x7fc7dc103450 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e4400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7dc112d80 con 0x7fc7dc102a90 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc19d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35842/0 (socket says 172.21.15.94:35842) 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e1000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7dc103450 0x7fc7dc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 -- 172.21.15.94:0/3000364322 learned_addr learned my addr 172.21.15.94:0/3000364322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e1a00700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc102a90 0x7fc7dc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 -- 172.21.15.94:0/3000364322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc102a90 msgr2=0x7fc7dc1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc102a90 0x7fc7dc1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 -- 172.21.15.94:0/3000364322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7dc103450 msgr2=0x7fc7dc199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7dc103450 0x7fc7dc199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e2400700 1 -- 172.21.15.94:0/3000364322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7dc19e100 con 0x7fc7dc108a50 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.582+0000 7fc7e1000700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7dc103450 0x7fc7dc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.583+0000 7fc7e2400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc19d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400ef30 tx=0x7fc7d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.583+0000 7fc7da000700 1 -- 172.21.15.94:0/3000364322 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d40092e0 con 0x7fc7dc108a50 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.583+0000 7fc7da000700 1 -- 172.21.15.94:0/3000364322 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7d4004bb0 con 0x7fc7dc108a50 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.583+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7dc19e390 con 0x7fc7dc108a50 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.583+0000 7fc7da000700 1 -- 172.21.15.94:0/3000364322 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4004020 con 0x7fc7dc108a50 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.583+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7dc074c70 con 0x7fc7dc108a50 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.584+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7dc104180 con 0x7fc7dc108a50 2024-04-25T15:00:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.587+0000 7fc7da000700 1 -- 172.21.15.94:0/3000364322 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc7d401b030 con 0x7fc7dc108a50 2024-04-25T15:00:53.570 INFO:teuthology.orchestra.run.smithi094.stdout:111669149766 2024-04-25T15:00:53.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.588+0000 7fc7da000700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc7c40a3cd0 0x7fc7c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:53.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.588+0000 7fc7da000700 1 -- 172.21.15.94:0/3000364322 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc7d40d09e0 con 0x7fc7dc108a50 2024-04-25T15:00:53.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.588+0000 7fc7e1a00700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc7c40a3cd0 0x7fc7c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:53.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.589+0000 7fc7e1a00700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc7c40a3cd0 0x7fc7c40a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00098e0 tx=0x7fc7d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.591+0000 7fc7da000700 1 -- 172.21.15.94:0/3000364322 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7d4022020 con 0x7fc7dc108a50 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.836+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc7dc02df80 con 0x7fc7dc108a50 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.836+0000 7fc7da000700 1 -- 172.21.15.94:0/3000364322 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc7d4010040 con 0x7fc7dc108a50 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.839+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc7c40a3cd0 msgr2=0x7fc7c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.839+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc7c40a3cd0 0x7fc7c40a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7d00098e0 tx=0x7fc7d0005960 comp rx=0 tx=0).stop 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.839+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 msgr2=0x7fc7dc19d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.839+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc19d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7d400ef30 tx=0x7fc7d400c3a0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.839+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 shutdown_connections 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.839+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc7c40a3cd0 0x7fc7c40a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.840+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7dc108a50 0x7fc7dc19d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.840+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7dc102a90 0x7fc7dc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.840+0000 7fc7e4400700 1 --2- 172.21.15.94:0/3000364322 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7dc103450 0x7fc7dc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.840+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 >> 172.21.15.94:0/3000364322 conn(0x7fc7dc079ba0 msgr2=0x7fc7dc101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.840+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 shutdown_connections 2024-04-25T15:00:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:52.840+0000 7fc7e4400700 1 -- 172.21.15.94:0/3000364322 wait complete. 2024-04-25T15:00:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:53 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3000364322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:00:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:53 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1118361759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T15:00:53.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:53 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3000364322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:00:53.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:53 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1118361759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T15:00:53.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:53 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3000364322' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:00:53.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:53 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1118361759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T15:00:54.006 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888124 got 146028888129 for osd.4 2024-04-25T15:00:54.007 DEBUG:teuthology.parallel:result is None 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 -- 172.21.15.94:0/3992773159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0109fc0 msgr2=0x7f0ae0111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/3992773159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0109fc0 0x7f0ae0111b90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800b0d0 tx=0x7f0ad800b3e0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 -- 172.21.15.94:0/3992773159 shutdown_connections 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/3992773159 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0109fc0 0x7f0ae0111b90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/3992773159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae0104ee0 0x7f0ae01052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/3992773159 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0105890 0x7f0ae01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 -- 172.21.15.94:0/3992773159 >> 172.21.15.94:0/3992773159 conn(0x7f0ae0100770 msgr2=0x7f0ae0102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 -- 172.21.15.94:0/3992773159 shutdown_connections 2024-04-25T15:00:54.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.212+0000 7f0ae6e00700 1 -- 172.21.15.94:0/3992773159 wait complete. 2024-04-25T15:00:54.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.213+0000 7f0ae6e00700 1 Processor -- start 2024-04-25T15:00:54.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.213+0000 7f0ae6e00700 1 -- start start 2024-04-25T15:00:54.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0ae6e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae0104ee0 0x7f0ae019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:54.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0ae6e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0105890 0x7f0ae019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:54.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0ae6e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0109fc0 0x7f0ae019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:54.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0ae6e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0114ec0 con 0x7f0ae0105890 2024-04-25T15:00:54.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0ae6e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0114d40 con 0x7f0ae0109fc0 2024-04-25T15:00:54.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0ae6e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0115040 con 0x7f0ae0104ee0 2024-04-25T15:00:54.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0adf400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0105890 0x7f0ae019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:54.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0adf400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0105890 0x7f0ae019b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35884/0 (socket says 172.21.15.94:35884) 2024-04-25T15:00:54.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0adf400700 1 -- 172.21.15.94:0/361186301 learned_addr learned my addr 172.21.15.94:0/361186301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0ae4e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0109fc0 0x7f0ae019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.214+0000 7f0adfe00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae0104ee0 0x7f0ae019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adf400700 1 -- 172.21.15.94:0/361186301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae0104ee0 msgr2=0x7f0ae019b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adf400700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae0104ee0 0x7f0ae019b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adf400700 1 -- 172.21.15.94:0/361186301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0109fc0 msgr2=0x7f0ae019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adf400700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0109fc0 0x7f0ae019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adf400700 1 -- 172.21.15.94:0/361186301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae01a0320 con 0x7f0ae0105890 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0ae4e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0109fc0 0x7f0ae019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:54.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adf400700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0105890 0x7f0ae019b850 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00e970 tx=0x7f0acc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:54.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adcc00700 1 -- 172.21.15.94:0/361186301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc004a30 con 0x7f0ae0105890 2024-04-25T15:00:54.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae01a0610 con 0x7f0ae0105890 2024-04-25T15:00:54.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adcc00700 1 -- 172.21.15.94:0/361186301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0acc00cc90 con 0x7f0ae0105890 2024-04-25T15:00:54.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.215+0000 7f0adcc00700 1 -- 172.21.15.94:0/361186301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc005610 con 0x7f0ae0105890 2024-04-25T15:00:54.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.216+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a7e90 con 0x7f0ae0105890 2024-04-25T15:00:54.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.217+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae0069730 con 0x7f0ae0105890 2024-04-25T15:00:54.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.219+0000 7f0adcc00700 1 -- 172.21.15.94:0/361186301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0acc00c7e0 con 0x7f0ae0105890 2024-04-25T15:00:54.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.220+0000 7f0adcc00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0ac40a3cd0 0x7f0ac40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:54.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.220+0000 7f0adcc00700 1 -- 172.21.15.94:0/361186301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0acc0d0690 con 0x7f0ae0105890 2024-04-25T15:00:54.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.220+0000 7f0adfe00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0ac40a3cd0 0x7f0ac40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:54.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.221+0000 7f0adfe00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0ac40a3cd0 0x7f0ac40a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009c60 tx=0x7f0ad0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:54.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.224+0000 7f0adcc00700 1 -- 172.21.15.94:0/361186301 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0acc09ced0 con 0x7f0ae0105890 2024-04-25T15:00:54.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.454+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0ae002df80 con 0x7f0ae0105890 2024-04-25T15:00:54.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.455+0000 7f0adcc00700 1 -- 172.21.15.94:0/361186301 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0acc09c7d0 con 0x7f0ae0105890 2024-04-25T15:00:54.456 INFO:teuthology.orchestra.run.smithi094.stdout:257698037803 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0ac40a3cd0 msgr2=0x7f0ac40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0ac40a3cd0 0x7f0ac40a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009c60 tx=0x7f0ad0009260 comp rx=0 tx=0).stop 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0105890 msgr2=0x7f0ae019b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0105890 0x7f0ae019b850 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00e970 tx=0x7f0acc00ec80 comp rx=0 tx=0).stop 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 shutdown_connections 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0ac40a3cd0 0x7f0ac40a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ae0109fc0 0x7f0ae019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ae0104ee0 0x7f0ae019b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 --2- 172.21.15.94:0/361186301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0105890 0x7f0ae019b850 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:54.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.458+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 >> 172.21.15.94:0/361186301 conn(0x7f0ae0100770 msgr2=0x7f0ae0101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:54.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.459+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 shutdown_connections 2024-04-25T15:00:54.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:54.459+0000 7f0ae6e00700 1 -- 172.21.15.94:0/361186301 wait complete. 2024-04-25T15:00:54.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:54 smithi094 ceph-mon[27862]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:54.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:54 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/361186301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T15:00:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:54 smithi138 ceph-mon[32875]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:54 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/361186301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T15:00:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:54 smithi137 ceph-mon[32439]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:54 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/361186301' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T15:00:55.053 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.332+0000 7f9ad6400700 1 -- 172.21.15.94:0/2099718699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0104dd0 msgr2=0x7f9ad01051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.332+0000 7f9ad6400700 1 --2- 172.21.15.94:0/2099718699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0104dd0 0x7f9ad01051b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40099c0 tx=0x7f9ac4009cd0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.333+0000 7f9ad6400700 1 -- 172.21.15.94:0/2099718699 shutdown_connections 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.333+0000 7f9ad6400700 1 --2- 172.21.15.94:0/2099718699 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0109eb0 0x7f9ad0111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.333+0000 7f9ad6400700 1 --2- 172.21.15.94:0/2099718699 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0104dd0 0x7f9ad01051b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.333+0000 7f9ad6400700 1 --2- 172.21.15.94:0/2099718699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ad0105780 0x7f9ad01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.333+0000 7f9ad6400700 1 -- 172.21.15.94:0/2099718699 >> 172.21.15.94:0/2099718699 conn(0x7f9ad0100660 msgr2=0x7f9ad0102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:55.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.333+0000 7f9ad6400700 1 -- 172.21.15.94:0/2099718699 shutdown_connections 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.333+0000 7f9ad6400700 1 -- 172.21.15.94:0/2099718699 wait complete. 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.334+0000 7f9ad6400700 1 Processor -- start 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.334+0000 7f9ad6400700 1 -- start start 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad6400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ad0104dd0 0x7f9ad019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad6400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0105780 0x7f9ad019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad6400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0109eb0 0x7f9ad019fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad6400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0114dc0 con 0x7f9ad0105780 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad6400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0114c40 con 0x7f9ad0109eb0 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad6400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0114f40 con 0x7f9ad0104dd0 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9acfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0105780 0x7f9ad019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9acfe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0105780 0x7f9ad019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35860/0 (socket says 172.21.15.94:35860) 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9acfe00700 1 -- 172.21.15.94:0/1118361759 learned_addr learned my addr 172.21.15.94:0/1118361759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:55.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad5000700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ad0104dd0 0x7f9ad019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.335+0000 7f9ad5a00700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0109eb0 0x7f9ad019fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acfe00700 1 -- 172.21.15.94:0/1118361759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ad0104dd0 msgr2=0x7f9ad019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acfe00700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ad0104dd0 0x7f9ad019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acfe00700 1 -- 172.21.15.94:0/1118361759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0109eb0 msgr2=0x7f9ad019fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acfe00700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0109eb0 0x7f9ad019fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acfe00700 1 -- 172.21.15.94:0/1118361759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad01a02a0 con 0x7f9ad0105780 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acfe00700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0105780 0x7f9ad019b7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9f0 tx=0x7f9abc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acd600700 1 -- 172.21.15.94:0/1118361759 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc004a30 con 0x7f9ad0105780 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acd600700 1 -- 172.21.15.94:0/1118361759 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9abc00cc70 con 0x7f9ad0105780 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9acd600700 1 -- 172.21.15.94:0/1118361759 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc005530 con 0x7f9ad0105780 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad01a0590 con 0x7f9ad0105780 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.336+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad01a7e10 con 0x7f9ad0105780 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.337+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad0069730 con 0x7f9ad0105780 2024-04-25T15:00:55.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.339+0000 7f9acd600700 1 -- 172.21.15.94:0/1118361759 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9abc004bd0 con 0x7f9ad0105780 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.341+0000 7f9acd600700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9ab80a3d50 0x7f9ab80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.341+0000 7f9acd600700 1 -- 172.21.15.94:0/1118361759 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f9abc0d0720 con 0x7f9ad0105780 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.341+0000 7f9ad5000700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9ab80a3d50 0x7f9ab80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.342+0000 7f9ad5000700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9ab80a3d50 0x7f9ab80a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.345+0000 7f9acd600700 1 -- 172.21.15.94:0/1118361759 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9abc00ce20 con 0x7f9ad0105780 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.606+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f9ad002df80 con 0x7f9ad0105780 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.606+0000 7f9acd600700 1 -- 172.21.15.94:0/1118361759 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f9abc09cf60 con 0x7f9ad0105780 2024-04-25T15:00:55.513 INFO:teuthology.orchestra.run.smithi094.stdout:347892351001 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9ab80a3d50 msgr2=0x7f9ab80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9ab80a3d50 0x7f9ab80a6210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac4005fb0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0105780 msgr2=0x7f9ad019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0105780 0x7f9ad019b7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9abc00e9f0 tx=0x7f9abc00ed00 comp rx=0 tx=0).stop 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 shutdown_connections 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f9ab80a3d50 0x7f9ab80a6210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f9ad0109eb0 0x7f9ad019fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9ad0104dd0 0x7f9ad019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 --2- 172.21.15.94:0/1118361759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ad0105780 0x7f9ad019b7d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 >> 172.21.15.94:0/1118361759 conn(0x7f9ad0100660 msgr2=0x7f9ad01029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:55.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 shutdown_connections 2024-04-25T15:00:55.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:53.609+0000 7f9ad6400700 1 -- 172.21.15.94:0/1118361759 wait complete. 2024-04-25T15:00:55.931 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149759 got 111669149766 for osd.3 2024-04-25T15:00:55.932 DEBUG:teuthology.parallel:result is None 2024-04-25T15:00:56.654 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037799 got 257698037803 for osd.8 2024-04-25T15:00:56.654 DEBUG:teuthology.parallel:result is None 2024-04-25T15:00:56.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:56 smithi094 ceph-mon[27862]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:56.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:56 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/739048547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:00:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:56 smithi138 ceph-mon[32875]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:56 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/739048547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:00:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:56 smithi137 ceph-mon[32439]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:00:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:56 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/739048547' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:00:57.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738454 for osd.0 2024-04-25T15:00:57.538 DEBUG:teuthology.parallel:result is None 2024-04-25T15:00:58.057 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:57 smithi094 ceph-mon[27862]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 -- 172.21.15.94:0/995038059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc074c70 msgr2=0x7ffabc075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:58.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 --2- 172.21.15.94:0/995038059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc074c70 0x7ffabc075050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009990 tx=0x7ffaac009ca0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 -- 172.21.15.94:0/995038059 shutdown_connections 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 --2- 172.21.15.94:0/995038059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 0x7ffabc111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 --2- 172.21.15.94:0/995038059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc074c70 0x7ffabc075050 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 --2- 172.21.15.94:0/995038059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffabc075590 0x7ffabc10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 -- 172.21.15.94:0/995038059 >> 172.21.15.94:0/995038059 conn(0x7ffabc079ba0 msgr2=0x7ffabc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 -- 172.21.15.94:0/995038059 shutdown_connections 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.359+0000 7ffac0c00700 1 -- 172.21.15.94:0/995038059 wait complete. 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.361+0000 7ffac0c00700 1 Processor -- start 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.361+0000 7ffac0c00700 1 -- start start 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffac0c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffabc074c70 0x7ffabc19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffac0c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc075590 0x7ffabc19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffac0c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 0x7ffabc19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:58.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffac0c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc114e60 con 0x7ffabc075590 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffac0c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc114ce0 con 0x7ffabc10f4a0 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffac0c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc114fe0 con 0x7ffabc074c70 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffaba800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 0x7ffabc19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffab9400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc075590 0x7ffabc19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffab9e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffabc074c70 0x7ffabc19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffaba800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 0x7ffabc19fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46200/0 (socket says 172.21.15.94:46200) 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.362+0000 7ffaba800700 1 -- 172.21.15.94:0/739048547 learned_addr learned my addr 172.21.15.94:0/739048547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffab9e00700 1 -- 172.21.15.94:0/739048547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 msgr2=0x7ffabc19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffab9e00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 0x7ffabc19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffab9e00700 1 -- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc075590 msgr2=0x7ffabc19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffab9e00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc075590 0x7ffabc19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffab9e00700 1 -- 172.21.15.94:0/739048547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc1a02d0 con 0x7ffabc074c70 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffab9400700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc075590 0x7ffabc19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:58.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffaba800700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 0x7ffabc19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.363+0000 7ffab9e00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffabc074c70 0x7ffabc19b310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0059d0 tx=0x7ffaac0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.364+0000 7ffab2a00700 1 -- 172.21.15.94:0/739048547 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac027070 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.364+0000 7ffab2a00700 1 -- 172.21.15.94:0/739048547 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac022470 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.364+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc1a0560 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.364+0000 7ffab2a00700 1 -- 172.21.15.94:0/739048547 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac017650 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.364+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a7e40 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.365+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc076330 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.367+0000 7ffab2a00700 1 -- 172.21.15.94:0/739048547 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffaac02a070 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.368+0000 7ffab2a00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffaa00a3cd0 0x7ffaa00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.368+0000 7ffab2a00700 1 -- 172.21.15.94:0/739048547 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7ffaac0d1650 con 0x7ffabc074c70 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.368+0000 7ffab9400700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffaa00a3cd0 0x7ffaa00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:58.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.369+0000 7ffab9400700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffaa00a3cd0 0x7ffaa00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffabc110ff0 tx=0x7ffaa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:58.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.371+0000 7ffab2a00700 1 -- 172.21.15.94:0/739048547 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffaac013070 con 0x7ffabc074c70 2024-04-25T15:00:58.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.602+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ffabc02df80 con 0x7ffabc074c70 2024-04-25T15:00:58.315 INFO:teuthology.orchestra.run.smithi094.stdout:77309411406 2024-04-25T15:00:58.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.603+0000 7ffab2a00700 1 -- 172.21.15.94:0/739048547 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffaac09df20 con 0x7ffabc074c70 2024-04-25T15:00:58.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffaa00a3cd0 msgr2=0x7ffaa00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:58.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffaa00a3cd0 0x7ffaa00a6190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffabc110ff0 tx=0x7ffaa8009260 comp rx=0 tx=0).stop 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffabc074c70 msgr2=0x7ffabc19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffabc074c70 0x7ffabc19b310 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0059d0 tx=0x7ffaac0047b0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 shutdown_connections 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7ffaa00a3cd0 0x7ffaa00a6190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ffabc10f4a0 0x7ffabc19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffabc074c70 0x7ffabc19b310 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.606+0000 7ffac0c00700 1 --2- 172.21.15.94:0/739048547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffabc075590 0x7ffabc19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.607+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 >> 172.21.15.94:0/739048547 conn(0x7ffabc079ba0 msgr2=0x7ffabc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.607+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 shutdown_connections 2024-04-25T15:00:58.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:56.607+0000 7ffac0c00700 1 -- 172.21.15.94:0/739048547 wait complete. 2024-04-25T15:00:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:57 smithi138 ceph-mon[32875]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:57 smithi137 ceph-mon[32439]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:00:59.048 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350998 got 347892351001 for osd.11 2024-04-25T15:00:59.048 DEBUG:teuthology.parallel:result is None 2024-04-25T15:00:59.244 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:00:59 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1741498396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:00:59.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:00:59 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1741498396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:00:59.473 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:00:59.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:00:59 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1741498396' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.797+0000 7fc983c00700 1 -- 172.21.15.94:0/1968453188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 msgr2=0x7fc97c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.797+0000 7fc983c00700 1 --2- 172.21.15.94:0/1968453188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c102f10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc9680098f0 tx=0x7fc968009c00 comp rx=0 tx=0).stop 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.798+0000 7fc983c00700 1 -- 172.21.15.94:0/1968453188 shutdown_connections 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.798+0000 7fc983c00700 1 --2- 172.21.15.94:0/1968453188 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc97c103450 0x7fc97c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.798+0000 7fc983c00700 1 --2- 172.21.15.94:0/1968453188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc97c108a50 0x7fc97c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.798+0000 7fc983c00700 1 --2- 172.21.15.94:0/1968453188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c102f10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.798+0000 7fc983c00700 1 -- 172.21.15.94:0/1968453188 >> 172.21.15.94:0/1968453188 conn(0x7fc97c079ba0 msgr2=0x7fc97c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:59.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.798+0000 7fc983c00700 1 -- 172.21.15.94:0/1968453188 shutdown_connections 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.798+0000 7fc983c00700 1 -- 172.21.15.94:0/1968453188 wait complete. 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 Processor -- start 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 -- start start 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc97c103450 0x7fc97c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc97c108a50 0x7fc97c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c112c50 con 0x7fc97c102a90 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c112ad0 con 0x7fc97c108a50 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.799+0000 7fc983c00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c112dd0 con 0x7fc97c103450 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c19b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49988/0 (socket says 172.21.15.94:49988) 2024-04-25T15:00:59.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 -- 172.21.15.94:0/1741498396 learned_addr learned my addr 172.21.15.94:0/1741498396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc97be00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc97c103450 0x7fc97c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981c00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc97c108a50 0x7fc97c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 -- 172.21.15.94:0/1741498396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc97c103450 msgr2=0x7fc97c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc97c103450 0x7fc97c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 -- 172.21.15.94:0/1741498396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc97c108a50 msgr2=0x7fc97c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc97c108a50 0x7fc97c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 -- 172.21.15.94:0/1741498396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c1a0490 con 0x7fc97c102a90 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.800+0000 7fc981200700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c19b480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e9f0 tx=0x7fc97000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.801+0000 7fc979600700 1 -- 172.21.15.94:0/1741498396 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970004a30 con 0x7fc97c102a90 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.801+0000 7fc979600700 1 -- 172.21.15.94:0/1741498396 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc97000cc70 con 0x7fc97c102a90 2024-04-25T15:00:59.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.801+0000 7fc979600700 1 -- 172.21.15.94:0/1741498396 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970005540 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.801+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c1a0780 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.801+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c1a8000 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.802+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c069730 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.804+0000 7fc979600700 1 -- 172.21.15.94:0/1741498396 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc970004bd0 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.805+0000 7fc979600700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc9640a3d50 0x7fc9640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.805+0000 7fc979600700 1 -- 172.21.15.94:0/1741498396 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc9700d0620 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.805+0000 7fc97be00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc9640a3d50 0x7fc9640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.806+0000 7fc97be00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc9640a3d50 0x7fc9640a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc9680098c0 tx=0x7fc968019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:58.810+0000 7fc979600700 1 -- 172.21.15.94:0/1741498396 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc97000ce20 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.042+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc97c19c7e0 con 0x7fc97c102a90 2024-04-25T15:00:59.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.043+0000 7fc979600700 1 -- 172.21.15.94:0/1741498396 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc97009ce60 con 0x7fc97c102a90 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stdout:55834574931 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc9640a3d50 msgr2=0x7fc9640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc9640a3d50 0x7fc9640a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc9680098c0 tx=0x7fc968019040 comp rx=0 tx=0).stop 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 msgr2=0x7fc97c19b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c19b480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc97000e9f0 tx=0x7fc97000ed00 comp rx=0 tx=0).stop 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 shutdown_connections 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc9640a3d50 0x7fc9640a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc97c108a50 0x7fc97c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc97c102a90 0x7fc97c19b480 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 --2- 172.21.15.94:0/1741498396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc97c103450 0x7fc97c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:00:59.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 >> 172.21.15.94:0/1741498396 conn(0x7fc97c079ba0 msgr2=0x7fc97c101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:00:59.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 shutdown_connections 2024-04-25T15:00:59.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.046+0000 7fc983c00700 1 -- 172.21.15.94:0/1741498396 wait complete. 2024-04-25T15:01:00.358 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:00 smithi094 ceph-mon[27862]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:00.359 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:00 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/452664166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T15:01:00.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:00 smithi138 ceph-mon[32875]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:00.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:00 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/452664166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T15:01:00.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:00 smithi137 ceph-mon[32439]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:00.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:00 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/452664166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T15:01:00.644 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574925 got 55834574931 for osd.1 2024-04-25T15:01:00.644 DEBUG:teuthology.parallel:result is None 2024-04-25T15:01:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 -- 172.21.15.94:0/472452822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe40075590 msgr2=0x7fbe4010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 --2- 172.21.15.94:0/472452822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe40075590 0x7fbe4010ef00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009990 tx=0x7fbe28009ca0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 -- 172.21.15.94:0/472452822 shutdown_connections 2024-04-25T15:01:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 --2- 172.21.15.94:0/472452822 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe4010f440 0x7fbe40111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 --2- 172.21.15.94:0/472452822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe40074c70 0x7fbe40075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 --2- 172.21.15.94:0/472452822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe40075590 0x7fbe4010ef00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 -- 172.21.15.94:0/472452822 >> 172.21.15.94:0/472452822 conn(0x7fbe40079ba0 msgr2=0x7fbe40079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 -- 172.21.15.94:0/472452822 shutdown_connections 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.734+0000 7fbe45400700 1 -- 172.21.15.94:0/472452822 wait complete. 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.735+0000 7fbe45400700 1 Processor -- start 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.735+0000 7fbe45400700 1 -- start start 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe45400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe40074c70 0x7fbe4019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe45400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe40075590 0x7fbe4019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe45400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe4010f440 0x7fbe4019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe45400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40114e50 con 0x7fbe4010f440 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe45400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40114cd0 con 0x7fbe40075590 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe45400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40114fd0 con 0x7fbe40074c70 2024-04-25T15:01:00.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe3e800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe40074c70 0x7fbe4019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe3e800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe40074c70 0x7fbe4019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:53876/0 (socket says 172.21.15.94:53876) 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe3e800700 1 -- 172.21.15.94:0/452664166 learned_addr learned my addr 172.21.15.94:0/452664166 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe3f200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe4010f440 0x7fbe4019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe35e00700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe40075590 0x7fbe4019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe35e00700 1 -- 172.21.15.94:0/452664166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe40074c70 msgr2=0x7fbe4019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe35e00700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe40074c70 0x7fbe4019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe35e00700 1 -- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe4010f440 msgr2=0x7fbe4019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe35e00700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe4010f440 0x7fbe4019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.736+0000 7fbe35e00700 1 -- 172.21.15.94:0/452664166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe401a0380 con 0x7fbe40075590 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.737+0000 7fbe3f200700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe4010f440 0x7fbe4019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.737+0000 7fbe35e00700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe40075590 0x7fbe4019b900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe28000c00 tx=0x7fbe280047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:00.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.737+0000 7fbe37e00700 1 -- 172.21.15.94:0/452664166 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28026070 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.737+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe401a0610 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.737+0000 7fbe37e00700 1 -- 172.21.15.94:0/452664166 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe28021410 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.737+0000 7fbe37e00700 1 -- 172.21.15.94:0/452664166 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28017650 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.737+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401a7ef0 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.738+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe40076360 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.740+0000 7fbe37e00700 1 -- 172.21.15.94:0/452664166 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe28029070 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.742+0000 7fbe37e00700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe240a3cd0 0x7fbe240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.742+0000 7fbe37e00700 1 -- 172.21.15.94:0/452664166 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbe280d13e0 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.742+0000 7fbe3e800700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe240a3cd0 0x7fbe240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.743+0000 7fbe3e800700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe240a3cd0 0x7fbe240a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009c10 tx=0x7fbe3000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.745+0000 7fbe37e00700 1 -- 172.21.15.94:0/452664166 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe28013070 con 0x7fbe40075590 2024-04-25T15:01:00.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.977+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbe4002df80 con 0x7fbe40075590 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stdout:201863462967 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.977+0000 7fbe37e00700 1 -- 172.21.15.94:0/452664166 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbe2809dcb0 con 0x7fbe40075590 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe240a3cd0 msgr2=0x7fbe240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe240a3cd0 0x7fbe240a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009c10 tx=0x7fbe3000b040 comp rx=0 tx=0).stop 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe40075590 msgr2=0x7fbe4019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe40075590 0x7fbe4019b900 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbe28000c00 tx=0x7fbe280047b0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 shutdown_connections 2024-04-25T15:01:00.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbe240a3cd0 0x7fbe240a6190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe4010f440 0x7fbe4019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe40074c70 0x7fbe4019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 --2- 172.21.15.94:0/452664166 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbe40075590 0x7fbe4019b900 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:00.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 >> 172.21.15.94:0/452664166 conn(0x7fbe40079ba0 msgr2=0x7fbe400ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:00.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 shutdown_connections 2024-04-25T15:01:00.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:00:59.981+0000 7fbe45400700 1 -- 172.21.15.94:0/452664166 wait complete. 2024-04-25T15:01:01.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 -- 172.21.15.94:0/3535812312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8410f440 msgr2=0x7fad84111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:01.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 --2- 172.21.15.94:0/3535812312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8410f440 0x7fad84111830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b0a0 tx=0x7fad7c00b3b0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 -- 172.21.15.94:0/3535812312 shutdown_connections 2024-04-25T15:01:01.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 --2- 172.21.15.94:0/3535812312 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad8410f440 0x7fad84111830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 --2- 172.21.15.94:0/3535812312 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad84074c70 0x7fad84075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 --2- 172.21.15.94:0/3535812312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad84075590 0x7fad8410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 -- 172.21.15.94:0/3535812312 >> 172.21.15.94:0/3535812312 conn(0x7fad84079ba0 msgr2=0x7fad84079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:01.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 -- 172.21.15.94:0/3535812312 shutdown_connections 2024-04-25T15:01:01.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.559+0000 7fad8c000700 1 -- 172.21.15.94:0/3535812312 wait complete. 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.560+0000 7fad8c000700 1 Processor -- start 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8c000700 1 -- start start 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8c000700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad84074c70 0x7fad8419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8c000700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad84075590 0x7fad8419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8c000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad8410f440 0x7fad8419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8c000700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fad84114e40 con 0x7fad84074c70 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8c000700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fad84114cc0 con 0x7fad8410f440 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8c000700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad84114fc0 con 0x7fad84075590 2024-04-25T15:01:01.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad89600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad84074c70 0x7fad8419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8a000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad8410f440 0x7fad8419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8a000700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad8410f440 0x7fad8419fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55870/0 (socket says 172.21.15.94:55870) 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad89600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad84074c70 0x7fad8419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50012/0 (socket says 172.21.15.94:50012) 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad8a000700 1 -- 172.21.15.94:0/2496796530 learned_addr learned my addr 172.21.15.94:0/2496796530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.561+0000 7fad88c00700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad84075590 0x7fad8419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.562+0000 7fad8a000700 1 -- 172.21.15.94:0/2496796530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad84075590 msgr2=0x7fad8419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.562+0000 7fad8a000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad84075590 0x7fad8419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.562+0000 7fad8a000700 1 -- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad84074c70 msgr2=0x7fad8419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.562+0000 7fad8a000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad84074c70 0x7fad8419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.562+0000 7fad8a000700 1 -- 172.21.15.94:0/2496796530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad841a0270 con 0x7fad8410f440 2024-04-25T15:01:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.562+0000 7fad8a000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad8410f440 0x7fad8419fb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b070 tx=0x7fad7c004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.562+0000 7fad82000700 1 -- 172.21.15.94:0/2496796530 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c021070 con 0x7fad8410f440 2024-04-25T15:01:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.563+0000 7fad82000700 1 -- 172.21.15.94:0/2496796530 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad7c007e30 con 0x7fad8410f440 2024-04-25T15:01:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.563+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad841a0500 con 0x7fad8410f440 2024-04-25T15:01:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.563+0000 7fad82000700 1 -- 172.21.15.94:0/2496796530 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0045e0 con 0x7fad8410f440 2024-04-25T15:01:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.563+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad841a7de0 con 0x7fad8410f440 2024-04-25T15:01:01.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.564+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad84069730 con 0x7fad8410f440 2024-04-25T15:01:01.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.566+0000 7fad82000700 1 -- 172.21.15.94:0/2496796530 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fad7c003c90 con 0x7fad8410f440 2024-04-25T15:01:01.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.567+0000 7fad82000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad6c0a3cd0 0x7fad6c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:01.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.567+0000 7fad89600700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad6c0a3cd0 0x7fad6c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:01.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.567+0000 7fad82000700 1 -- 172.21.15.94:0/2496796530 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fad7c00f020 con 0x7fad8410f440 2024-04-25T15:01:01.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.568+0000 7fad89600700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad6c0a3cd0 0x7fad6c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad78009680 tx=0x7fad78009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:01.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.571+0000 7fad82000700 1 -- 172.21.15.94:0/2496796530 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad7c02a020 con 0x7fad8410f440 2024-04-25T15:01:01.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.808+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fad8402df80 con 0x7fad8410f440 2024-04-25T15:01:01.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.809+0000 7fad82000700 1 -- 172.21.15.94:0/2496796530 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fad7c09e210 con 0x7fad8410f440 2024-04-25T15:01:01.811 INFO:teuthology.orchestra.run.smithi094.stdout:313532612640 2024-04-25T15:01:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad6c0a3cd0 msgr2=0x7fad6c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:01.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad6c0a3cd0 0x7fad6c0a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad78009680 tx=0x7fad78009260 comp rx=0 tx=0).stop 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad8410f440 msgr2=0x7fad8419fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad8410f440 0x7fad8419fb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00b070 tx=0x7fad7c004c60 comp rx=0 tx=0).stop 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 shutdown_connections 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fad6c0a3cd0 0x7fad6c0a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fad8410f440 0x7fad8419fb30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad84074c70 0x7fad8419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 --2- 172.21.15.94:0/2496796530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad84075590 0x7fad8419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.812+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 >> 172.21.15.94:0/2496796530 conn(0x7fad84079ba0 msgr2=0x7fad840ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.813+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 shutdown_connections 2024-04-25T15:01:01.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:01.813+0000 7fad8c000700 1 -- 172.21.15.94:0/2496796530 wait complete. 2024-04-25T15:01:02.839 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:02 smithi094 ceph-mon[27862]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:02.839 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:02 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2496796530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T15:01:02.884 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:02 smithi137 ceph-mon[32439]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:02.884 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:02 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2496796530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T15:01:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:02 smithi138 ceph-mon[32875]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:02 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2496796530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T15:01:03.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:03 smithi094 ceph-mon[27862]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T15:01:03.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:03 smithi138 ceph-mon[32875]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T15:01:03.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:03 smithi137 ceph-mon[32439]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T15:01:03.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411401 got 77309411406 for osd.2 2024-04-25T15:01:03.989 DEBUG:teuthology.parallel:result is None 2024-04-25T15:01:04.885 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:04 smithi094 ceph-mon[27862]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:04.938 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612635 got 313532612640 for osd.10 2024-04-25T15:01:04.961 DEBUG:teuthology.parallel:result is None 2024-04-25T15:01:04.961 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:04 smithi138 ceph-mon[32875]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:04 smithi137 ceph-mon[32439]: pgmap v349: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:05.178 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462961 got 201863462967 for osd.6 2024-04-25T15:01:05.178 DEBUG:teuthology.parallel:result is None 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 -- 172.21.15.94:0/1904428816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c1092d0 msgr2=0x7f1d5c1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 --2- 172.21.15.94:0/1904428816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1096b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1d500099c0 tx=0x7f1d50009cd0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 -- 172.21.15.94:0/1904428816 shutdown_connections 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 --2- 172.21.15.94:0/1904428816 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c10e3b0 0x7f1d5c115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 --2- 172.21.15.94:0/1904428816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1096b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 --2- 172.21.15.94:0/1904428816 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d5c109c80 0x7f1d5c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 -- 172.21.15.94:0/1904428816 >> 172.21.15.94:0/1904428816 conn(0x7f1d5c076210 msgr2=0x7f1d5c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 -- 172.21.15.94:0/1904428816 shutdown_connections 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.703+0000 7f1d64400700 1 -- 172.21.15.94:0/1904428816 wait complete. 2024-04-25T15:01:06.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.704+0000 7f1d64400700 1 Processor -- start 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d64400700 1 -- start start 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d64400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d64400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d5c109c80 0x7f1d5c1a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d64400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c10e3b0 0x7f1d5c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d64400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c1192b0 con 0x7f1d5c10e3b0 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d64400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c119130 con 0x7f1d5c1092d0 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d64400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c119430 con 0x7f1d5c109c80 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d61a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d62400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c10e3b0 0x7f1d5c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d61a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:55898/0 (socket says 172.21.15.94:55898) 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d61a00700 1 -- 172.21.15.94:0/667304250 learned_addr learned my addr 172.21.15.94:0/667304250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.705+0000 7f1d61000700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d5c109c80 0x7f1d5c1a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d61000700 1 -- 172.21.15.94:0/667304250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c1092d0 msgr2=0x7f1d5c1a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d61000700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d61000700 1 -- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c10e3b0 msgr2=0x7f1d5c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d61000700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c10e3b0 0x7f1d5c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d61000700 1 -- 172.21.15.94:0/667304250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c19fd50 con 0x7f1d5c109c80 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d61a00700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d62400700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c10e3b0 0x7f1d5c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d61000700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d5c109c80 0x7f1d5c1a5920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e9c0 tx=0x7f1d4800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.706+0000 7f1d5a000700 1 -- 172.21.15.94:0/667304250 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48004220 con 0x7f1d5c109c80 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.707+0000 7f1d5a000700 1 -- 172.21.15.94:0/667304250 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d480043c0 con 0x7f1d5c109c80 2024-04-25T15:01:06.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.707+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c1a0040 con 0x7f1d5c109c80 2024-04-25T15:01:06.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.707+0000 7f1d5a000700 1 -- 172.21.15.94:0/667304250 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48005020 con 0x7f1d5c109c80 2024-04-25T15:01:06.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.707+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c1ac2b0 con 0x7f1d5c109c80 2024-04-25T15:01:06.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.708+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5c069730 con 0x7f1d5c109c80 2024-04-25T15:01:06.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.710+0000 7f1d5a000700 1 -- 172.21.15.94:0/667304250 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1d48004750 con 0x7f1d5c109c80 2024-04-25T15:01:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.711+0000 7f1d5a000700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1d440a3c40 0x7f1d440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.711+0000 7f1d5a000700 1 -- 172.21.15.94:0/667304250 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1d480d0390 con 0x7f1d5c109c80 2024-04-25T15:01:06.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.711+0000 7f1d61a00700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1d440a3c40 0x7f1d440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.712+0000 7f1d61a00700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1d440a3c40 0x7f1d440a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000c00 tx=0x7f1d50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:06.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.716+0000 7f1d5a000700 1 -- 172.21.15.94:0/667304250 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d4809cd30 con 0x7f1d5c109c80 2024-04-25T15:01:06.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:06 smithi094 ceph-mon[27862]: pgmap v350: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:01:06.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.944+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f1d5c1a0ad0 con 0x7f1d5c109c80 2024-04-25T15:01:06.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.945+0000 7f1d5a000700 1 -- 172.21.15.94:0/667304250 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d4809c630 con 0x7f1d5c109c80 2024-04-25T15:01:06.947 INFO:teuthology.orchestra.run.smithi094.stdout:167503724605 2024-04-25T15:01:06.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1d440a3c40 msgr2=0x7f1d440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1d440a3c40 0x7f1d440a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d50000c00 tx=0x7f1d50005fd0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d5c109c80 msgr2=0x7f1d5c1a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d5c109c80 0x7f1d5c1a5920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e9c0 tx=0x7f1d4800ecd0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 shutdown_connections 2024-04-25T15:01:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1d440a3c40 0x7f1d440a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d5c10e3b0 0x7f1d5c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1d5c1092d0 0x7f1d5c1a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 --2- 172.21.15.94:0/667304250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1d5c109c80 0x7f1d5c1a5920 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.949+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 >> 172.21.15.94:0/667304250 conn(0x7f1d5c076210 msgr2=0x7f1d5c077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:06.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.950+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 shutdown_connections 2024-04-25T15:01:06.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.950+0000 7f1d64400700 1 -- 172.21.15.94:0/667304250 wait complete. 2024-04-25T15:01:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:06 smithi138 ceph-mon[32875]: pgmap v350: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 -- 172.21.15.94:0/691447470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e01044a0 msgr2=0x7f93e010c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 --2- 172.21.15.94:0/691447470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e01044a0 0x7f93e010c990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009990 tx=0x7f93cc009ca0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 -- 172.21.15.94:0/691447470 shutdown_connections 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 --2- 172.21.15.94:0/691447470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e010cfb0 0x7f93e010f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 --2- 172.21.15.94:0/691447470 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93e0103b80 0x7f93e0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 --2- 172.21.15.94:0/691447470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e01044a0 0x7f93e010c990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 -- 172.21.15.94:0/691447470 >> 172.21.15.94:0/691447470 conn(0x7f93e00fd3d0 msgr2=0x7f93e00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 -- 172.21.15.94:0/691447470 shutdown_connections 2024-04-25T15:01:06.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.959+0000 7f93e5800700 1 -- 172.21.15.94:0/691447470 wait complete. 2024-04-25T15:01:06.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 Processor -- start 2024-04-25T15:01:06.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 -- start start 2024-04-25T15:01:06.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93e0103b80 0x7f93e0104a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e01044a0 0x7f93e0104f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e010cfb0 0x7f93e0105480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0112bd0 con 0x7f93e010cfb0 2024-04-25T15:01:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0112a50 con 0x7f93e0103b80 2024-04-25T15:01:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.961+0000 7f93e5800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93e0112d50 con 0x7f93e01044a0 2024-04-25T15:01:06.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.962+0000 7f93df400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e01044a0 0x7f93e0104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.962+0000 7f93dfe00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93e0103b80 0x7f93e0104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.962+0000 7f93df400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e01044a0 0x7f93e0104f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35824/0 (socket says 172.21.15.94:35824) 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.962+0000 7f93df400700 1 -- 172.21.15.94:0/3901244733 learned_addr learned my addr 172.21.15.94:0/3901244733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.962+0000 7f93e4e00700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e010cfb0 0x7f93e0105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.963+0000 7f93df400700 1 -- 172.21.15.94:0/3901244733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93e0103b80 msgr2=0x7f93e0104a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.963+0000 7f93df400700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93e0103b80 0x7f93e0104a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.963+0000 7f93df400700 1 -- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e010cfb0 msgr2=0x7f93e0105480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.963+0000 7f93df400700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e010cfb0 0x7f93e0105480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:06.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.963+0000 7f93df400700 1 -- 172.21.15.94:0/3901244733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e019b1e0 con 0x7f93e01044a0 2024-04-25T15:01:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.963+0000 7f93df400700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e01044a0 0x7f93e0104f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000c00 tx=0x7f93cc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.964+0000 7f93dcc00700 1 -- 172.21.15.94:0/3901244733 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00fb40 con 0x7f93e01044a0 2024-04-25T15:01:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.964+0000 7f93dcc00700 1 -- 172.21.15.94:0/3901244733 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93cc00fce0 con 0x7f93e01044a0 2024-04-25T15:01:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.964+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e019b470 con 0x7f93e01044a0 2024-04-25T15:01:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.964+0000 7f93dcc00700 1 -- 172.21.15.94:0/3901244733 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc0053e0 con 0x7f93e01044a0 2024-04-25T15:01:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.964+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e019b8f0 con 0x7f93e01044a0 2024-04-25T15:01:06.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.965+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e006b890 con 0x7f93e01044a0 2024-04-25T15:01:06.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.968+0000 7f93dcc00700 1 -- 172.21.15.94:0/3901244733 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f93cc009d10 con 0x7f93e01044a0 2024-04-25T15:01:06.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.969+0000 7f93dcc00700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93c40a3dc0 0x7f93c40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:06.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.969+0000 7f93dcc00700 1 -- 172.21.15.94:0/3901244733 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f93cc0d12f0 con 0x7f93e01044a0 2024-04-25T15:01:06.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.969+0000 7f93dfe00700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93c40a3dc0 0x7f93c40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:06.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.970+0000 7f93dfe00700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93c40a3dc0 0x7f93c40a6280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009c60 tx=0x7f93d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:06.971+0000 7f93dcc00700 1 -- 172.21.15.94:0/3901244733 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93cc013070 con 0x7f93e01044a0 2024-04-25T15:01:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:06 smithi137 ceph-mon[32439]: pgmap v350: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:01:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.216+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f93e002df80 con 0x7f93e01044a0 2024-04-25T15:01:07.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.216+0000 7f93dcc00700 1 -- 172.21.15.94:0/3901244733 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f93cc09dec0 con 0x7f93e01044a0 2024-04-25T15:01:07.219 INFO:teuthology.orchestra.run.smithi094.stdout:223338299442 2024-04-25T15:01:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93c40a3dc0 msgr2=0x7f93c40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93c40a3dc0 0x7f93c40a6280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009c60 tx=0x7f93d0009260 comp rx=0 tx=0).stop 2024-04-25T15:01:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e01044a0 msgr2=0x7f93e0104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e01044a0 0x7f93e0104f40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000c00 tx=0x7f93cc0047b0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 shutdown_connections 2024-04-25T15:01:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93c40a3dc0 0x7f93c40a6280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93e010cfb0 0x7f93e0105480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93e0103b80 0x7f93e0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 --2- 172.21.15.94:0/3901244733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e01044a0 0x7f93e0104f40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.219+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 >> 172.21.15.94:0/3901244733 conn(0x7f93e00fd3d0 msgr2=0x7f93e00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.220+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 shutdown_connections 2024-04-25T15:01:07.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.220+0000 7f93e5800700 1 -- 172.21.15.94:0/3901244733 wait complete. 2024-04-25T15:01:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:07 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/667304250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:01:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:07 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3901244733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T15:01:07.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:07 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/667304250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:01:07.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:07 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3901244733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T15:01:07.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 -- 172.21.15.94:0/918954494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054075590 msgr2=0x7f005410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 --2- 172.21.15.94:0/918954494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054075590 0x7f005410ef00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f00400098f0 tx=0x7f0040009c00 comp rx=0 tx=0).stop 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 -- 172.21.15.94:0/918954494 shutdown_connections 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 --2- 172.21.15.94:0/918954494 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005410f440 0x7f0054111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 --2- 172.21.15.94:0/918954494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054074c70 0x7f0054075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 --2- 172.21.15.94:0/918954494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054075590 0x7f005410ef00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 -- 172.21.15.94:0/918954494 >> 172.21.15.94:0/918954494 conn(0x7f0054079ba0 msgr2=0x7f0054079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 -- 172.21.15.94:0/918954494 shutdown_connections 2024-04-25T15:01:07.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.975+0000 7f0058a00700 1 -- 172.21.15.94:0/918954494 wait complete. 2024-04-25T15:01:07.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.976+0000 7f0058a00700 1 Processor -- start 2024-04-25T15:01:07.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.976+0000 7f0058a00700 1 -- start start 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.976+0000 7f0058a00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054074c70 0x7f005419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.976+0000 7f0058a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054075590 0x7f005419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.976+0000 7f0058a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005410f440 0x7f005419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0058a00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0054114e70 con 0x7f0054075590 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0058a00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0054114cf0 con 0x7f005410f440 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0058a00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0054114ff0 con 0x7f0054074c70 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054074c70 0x7f005419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054074c70 0x7f005419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:35838/0 (socket says 172.21.15.94:35838) 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054075590 0x7f005419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 -- 172.21.15.94:0/3792109112 learned_addr learned my addr 172.21.15.94:0/3792109112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0052800700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005410f440 0x7f005419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 -- 172.21.15.94:0/3792109112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005410f440 msgr2=0x7f005419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005410f440 0x7f005419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 -- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054075590 msgr2=0x7f005419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:07.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054075590 0x7f005419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:07.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.977+0000 7f0051e00700 1 -- 172.21.15.94:0/3792109112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00541a0360 con 0x7f0054074c70 2024-04-25T15:01:07.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.978+0000 7f0051e00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054074c70 0x7f005419b3a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f004400e9a0 tx=0x7f004400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.978+0000 7f004aa00700 1 -- 172.21.15.94:0/3792109112 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004400cbd0 con 0x7f0054074c70 2024-04-25T15:01:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.978+0000 7f004aa00700 1 -- 172.21.15.94:0/3792109112 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f004400cd70 con 0x7f0054074c70 2024-04-25T15:01:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.978+0000 7f004aa00700 1 -- 172.21.15.94:0/3792109112 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044018680 con 0x7f0054074c70 2024-04-25T15:01:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.978+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00541a0650 con 0x7f0054074c70 2024-04-25T15:01:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.978+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00541a7ed0 con 0x7f0054074c70 2024-04-25T15:01:07.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.979+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0054076360 con 0x7f0054074c70 2024-04-25T15:01:07.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:07 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/667304250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:01:07.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:07 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3901244733' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T15:01:07.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.981+0000 7f004aa00700 1 -- 172.21.15.94:0/3792109112 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0044010430 con 0x7f0054074c70 2024-04-25T15:01:07.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.982+0000 7f004aa00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00380a3cc0 0x7f00380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.982+0000 7f004aa00700 1 -- 172.21.15.94:0/3792109112 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f00440d05e0 con 0x7f0054074c70 2024-04-25T15:01:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.982+0000 7f0051400700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00380a3cc0 0x7f00380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:07.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.983+0000 7f0051400700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00380a3cc0 0x7f00380a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0040009fd0 tx=0x7f0040019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:07.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:07.986+0000 7f004aa00700 1 -- 172.21.15.94:0/3792109112 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f004409cf80 con 0x7f0054074c70 2024-04-25T15:01:08.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.219+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f005402df80 con 0x7f0054074c70 2024-04-25T15:01:08.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.219+0000 7f004aa00700 1 -- 172.21.15.94:0/3792109112 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f004409c880 con 0x7f0054074c70 2024-04-25T15:01:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:292057776168 2024-04-25T15:01:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00380a3cc0 msgr2=0x7f00380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00380a3cc0 0x7f00380a6180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0040009fd0 tx=0x7f0040019040 comp rx=0 tx=0).stop 2024-04-25T15:01:08.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054074c70 msgr2=0x7f005419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054074c70 0x7f005419b3a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f004400e9a0 tx=0x7f004400ecb0 comp rx=0 tx=0).stop 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 shutdown_connections 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f00380a3cc0 0x7f00380a6180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f005410f440 0x7f005419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0054074c70 0x7f005419b3a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 --2- 172.21.15.94:0/3792109112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0054075590 0x7f005419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 >> 172.21.15.94:0/3792109112 conn(0x7f0054079ba0 msgr2=0x7f00540ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:08.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 shutdown_connections 2024-04-25T15:01:08.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:08.223+0000 7f0058a00700 1 -- 172.21.15.94:0/3792109112 wait complete. 2024-04-25T15:01:08.778 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:08 smithi094 ceph-mon[27862]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:08.779 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:08 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3792109112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T15:01:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:08 smithi138 ceph-mon[32875]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:08 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3792109112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T15:01:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:08 smithi137 ceph-mon[32439]: pgmap v351: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:08 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3792109112' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T15:01:09.673 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299437 got 223338299442 for osd.7 2024-04-25T15:01:09.673 DEBUG:teuthology.parallel:result is None 2024-04-25T15:01:10.040 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724599 got 167503724605 for osd.5 2024-04-25T15:01:10.040 DEBUG:teuthology.parallel:result is None 2024-04-25T15:01:10.571 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776162 got 292057776168 for osd.9 2024-04-25T15:01:10.571 DEBUG:teuthology.parallel:result is None 2024-04-25T15:01:10.571 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T15:01:10.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:01:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:10 smithi138 ceph-mon[32875]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:10 smithi094 ceph-mon[27862]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:10 smithi137 ceph-mon[32439]: pgmap v352: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:11.006 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:12.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:12.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:12 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:12.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:12 smithi138 ceph-mon[32875]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:12.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:12.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:12 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:12.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:12 smithi094 ceph-mon[27862]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:12.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:12.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:12 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:12.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:12 smithi137 ceph-mon[32439]: pgmap v353: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:13.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.264+0000 7fc7ae800700 1 -- 172.21.15.94:0/3157868660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806b0f0 msgr2=0x7fc7a810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.264+0000 7fc7ae800700 1 --2- 172.21.15.94:0/3157868660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806b0f0 0x7fc7a810f900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009990 tx=0x7fc7a0009ca0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.265+0000 7fc7ae800700 1 -- 172.21.15.94:0/3157868660 shutdown_connections 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.265+0000 7fc7ae800700 1 --2- 172.21.15.94:0/3157868660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806b0f0 0x7fc7a810f900 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.265+0000 7fc7ae800700 1 --2- 172.21.15.94:0/3157868660 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7a8108a30 0x7fc7a8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.265+0000 7fc7ae800700 1 --2- 172.21.15.94:0/3157868660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a806a750 0x7fc7a806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.265+0000 7fc7ae800700 1 -- 172.21.15.94:0/3157868660 >> 172.21.15.94:0/3157868660 conn(0x7fc7a80788f0 msgr2=0x7fc7a8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.265+0000 7fc7ae800700 1 -- 172.21.15.94:0/3157868660 shutdown_connections 2024-04-25T15:01:13.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.265+0000 7fc7ae800700 1 -- 172.21.15.94:0/3157868660 wait complete. 2024-04-25T15:01:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.266+0000 7fc7ae800700 1 Processor -- start 2024-04-25T15:01:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.266+0000 7fc7ae800700 1 -- start start 2024-04-25T15:01:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.266+0000 7fc7ae800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806a750 0x7fc7a819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7ae800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7a806b0f0 0x7fc7a819b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7ae800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8108a30 0x7fc7a819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7ae800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8112c60 con 0x7fc7a806a750 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7ae800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8112ae0 con 0x7fc7a806b0f0 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7ae800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8112de0 con 0x7fc7a8108a30 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806a750 0x7fc7a819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806a750 0x7fc7a819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53330/0 (socket says 172.21.15.94:53330) 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 -- 172.21.15.94:0/1795599700 learned_addr learned my addr 172.21.15.94:0/1795599700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc79ea00700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7a806b0f0 0x7fc7a819b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:13.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7e00700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8108a30 0x7fc7a819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 -- 172.21.15.94:0/1795599700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8108a30 msgr2=0x7fc7a819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8108a30 0x7fc7a819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 -- 172.21.15.94:0/1795599700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7a806b0f0 msgr2=0x7fc7a819b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7a806b0f0 0x7fc7a819b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.267+0000 7fc7a7400700 1 -- 172.21.15.94:0/1795599700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a81a03e0 con 0x7fc7a806a750 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.268+0000 7fc7a7400700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806a750 0x7fc7a819b3d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc79800ef10 tx=0x7fc79800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.268+0000 7fc7a4c00700 1 -- 172.21.15.94:0/1795599700 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79800c9c0 con 0x7fc7a806a750 2024-04-25T15:01:13.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.268+0000 7fc7a4c00700 1 -- 172.21.15.94:0/1795599700 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc798019070 con 0x7fc7a806a750 2024-04-25T15:01:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.268+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a81a0670 con 0x7fc7a806a750 2024-04-25T15:01:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.268+0000 7fc7a4c00700 1 -- 172.21.15.94:0/1795599700 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798014480 con 0x7fc7a806a750 2024-04-25T15:01:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.268+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a81a7f50 con 0x7fc7a806a750 2024-04-25T15:01:13.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.270+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a8069730 con 0x7fc7a806a750 2024-04-25T15:01:13.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.271+0000 7fc7a4c00700 1 -- 172.21.15.94:0/1795599700 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc798012050 con 0x7fc7a806a750 2024-04-25T15:01:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.273+0000 7fc7a4c00700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc78c0a3cc0 0x7fc78c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.273+0000 7fc7a4c00700 1 -- 172.21.15.94:0/1795599700 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc7980d0260 con 0x7fc7a806a750 2024-04-25T15:01:13.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.273+0000 7fc79ea00700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc78c0a3cc0 0x7fc78c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:13.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.274+0000 7fc79ea00700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc78c0a3cc0 0x7fc78c0a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8078df0 tx=0x7fc790009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:13.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.278+0000 7fc7a4c00700 1 -- 172.21.15.94:0/1795599700 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc798010040 con 0x7fc7a806a750 2024-04-25T15:01:13.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:13.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:13.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:13 smithi138 ceph-mon[32875]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:13.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:13.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:13.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:13 smithi094 ceph-mon[27862]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:13.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:01:13.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:01:13.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:13 smithi137 ceph-mon[32439]: from='mgr.14152 172.21.15.94:0/2533131949' entity='mgr.x' 2024-04-25T15:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.504+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7a819c730 con 0x7fc78c0a3cc0 2024-04-25T15:01:13.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.528+0000 7fc7a4c00700 1 -- 172.21.15.94:0/1795599700 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258749 (secure 0 0 0) 0x7fc7a819c730 con 0x7fc78c0a3cc0 2024-04-25T15:01:13.536 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:13.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc78c0a3cc0 msgr2=0x7fc78c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc78c0a3cc0 0x7fc78c0a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8078df0 tx=0x7fc790009260 comp rx=0 tx=0).stop 2024-04-25T15:01:13.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806a750 msgr2=0x7fc7a819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806a750 0x7fc7a819b3d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc79800ef10 tx=0x7fc79800c3a0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 shutdown_connections 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc78c0a3cc0 0x7fc78c0a6180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8108a30 0x7fc7a819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc7a806a750 0x7fc7a819b3d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 --2- 172.21.15.94:0/1795599700 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc7a806b0f0 0x7fc7a819b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 >> 172.21.15.94:0/1795599700 conn(0x7fc7a80788f0 msgr2=0x7fc7a810dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:13.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 shutdown_connections 2024-04-25T15:01:13.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:13.535+0000 7fc7ae800700 1 -- 172.21.15.94:0/1795599700 wait complete. 2024-04-25T15:01:13.540 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T15:01:14.196 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:14 smithi094 ceph-mon[27862]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:01:14.196 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:14 smithi094 ceph-mon[27862]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:14.260 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-04-25T15:01:11.508157+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652695,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":304,"num_read_kb":265,"num_write":263,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":707,"ondisk_log_size":707,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379004,"kb_used_data":13212,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562820,"statfs":{"total":1151940427776,"available":1151552327680,"internally_reserved":0,"allocated":13529088,"data_stored":7463121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004825"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519590+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:01:01.519590+0000","last_peered":"2024-04-25T15:01:01.519590+0000","last_clean":"2024-04-25T15:01:01.519590+0000","last_became_active":"2024-04-25T14:59:46.311856+0000","last_became_peered":"2024-04-25T14:59:46.311856+0000","last_unstale":"2024-04-25T15:01:01.519590+0000","last_undegraded":"2024-04-25T15:01:01.519590+0000","last_fullsized":"2024-04-25T15:01:01.519590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:07.469645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.514143+0000","last_change":"2024-04-25T14:59:46.310292+0000","last_active":"2024-04-25T15:01:01.514143+0000","last_peered":"2024-04-25T15:01:01.514143+0000","last_clean":"2024-04-25T15:01:01.514143+0000","last_became_active":"2024-04-25T14:59:46.309812+0000","last_became_peered":"2024-04-25T14:59:46.309812+0000","last_unstale":"2024-04-25T15:01:01.514143+0000","last_undegraded":"2024-04-25T15:01:01.514143+0000","last_fullsized":"2024-04-25T15:01:01.514143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:52.448406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515414+0000","last_change":"2024-04-25T14:59:46.300987+0000","last_active":"2024-04-25T15:01:01.515414+0000","last_peered":"2024-04-25T15:01:01.515414+0000","last_clean":"2024-04-25T15:01:01.515414+0000","last_became_active":"2024-04-25T14:59:46.299866+0000","last_became_peered":"2024-04-25T14:59:46.299866+0000","last_unstale":"2024-04-25T15:01:01.515414+0000","last_undegraded":"2024-04-25T15:01:01.515414+0000","last_fullsized":"2024-04-25T15:01:01.515414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:09.831131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.484945+0000","last_change":"2024-04-25T14:59:46.304278+0000","last_active":"2024-04-25T15:01:01.484945+0000","last_peered":"2024-04-25T15:01:01.484945+0000","last_clean":"2024-04-25T15:01:01.484945+0000","last_became_active":"2024-04-25T14:59:46.303956+0000","last_became_peered":"2024-04-25T14:59:46.303956+0000","last_unstale":"2024-04-25T15:01:01.484945+0000","last_undegraded":"2024-04-25T15:01:01.484945+0000","last_fullsized":"2024-04-25T15:01:01.484945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.752548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.503892+0000","last_change":"2024-04-25T14:59:46.323730+0000","last_active":"2024-04-25T15:01:01.503892+0000","last_peered":"2024-04-25T15:01:01.503892+0000","last_clean":"2024-04-25T15:01:01.503892+0000","last_became_active":"2024-04-25T14:59:46.323383+0000","last_became_peered":"2024-04-25T14:59:46.323383+0000","last_unstale":"2024-04-25T15:01:01.503892+0000","last_undegraded":"2024-04-25T15:01:01.503892+0000","last_fullsized":"2024-04-25T15:01:01.503892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.846392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506646+0000","last_change":"2024-04-25T14:59:46.317623+0000","last_active":"2024-04-25T15:01:01.506646+0000","last_peered":"2024-04-25T15:01:01.506646+0000","last_clean":"2024-04-25T15:01:01.506646+0000","last_became_active":"2024-04-25T14:59:46.317255+0000","last_became_peered":"2024-04-25T14:59:46.317255+0000","last_unstale":"2024-04-25T15:01:01.506646+0000","last_undegraded":"2024-04-25T15:01:01.506646+0000","last_fullsized":"2024-04-25T15:01:01.506646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:24.449615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.487029+0000","last_change":"2024-04-25T15:00:02.723289+0000","last_active":"2024-04-25T15:01:01.487029+0000","last_peered":"2024-04-25T15:01:01.487029+0000","last_clean":"2024-04-25T15:01:01.487029+0000","last_became_active":"2024-04-25T15:00:02.722968+0000","last_became_peered":"2024-04-25T15:00:02.722968+0000","last_unstale":"2024-04-25T15:01:01.487029+0000","last_undegraded":"2024-04-25T15:01:01.487029+0000","last_fullsized":"2024-04-25T15:01:01.487029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:45.674905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508192+0000","last_change":"2024-04-25T14:59:46.327687+0000","last_active":"2024-04-25T15:01:01.508192+0000","last_peered":"2024-04-25T15:01:01.508192+0000","last_clean":"2024-04-25T15:01:01.508192+0000","last_became_active":"2024-04-25T14:59:46.325905+0000","last_became_peered":"2024-04-25T14:59:46.325905+0000","last_unstale":"2024-04-25T15:01:01.508192+0000","last_undegraded":"2024-04-25T15:01:01.508192+0000","last_fullsized":"2024-04-25T15:01:01.508192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:50.058801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.505083+0000","last_change":"2024-04-25T14:59:46.304722+0000","last_active":"2024-04-25T15:01:01.505083+0000","last_peered":"2024-04-25T15:01:01.505083+0000","last_clean":"2024-04-25T15:01:01.505083+0000","last_became_active":"2024-04-25T14:59:46.304391+0000","last_became_peered":"2024-04-25T14:59:46.304391+0000","last_unstale":"2024-04-25T15:01:01.505083+0000","last_undegraded":"2024-04-25T15:01:01.505083+0000","last_fullsized":"2024-04-25T15:01:01.505083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:15.735670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.487719+0000","last_change":"2024-04-25T14:59:46.307796+0000","last_active":"2024-04-25T15:01:01.487719+0000","last_peered":"2024-04-25T15:01:01.487719+0000","last_clean":"2024-04-25T15:01:01.487719+0000","last_became_active":"2024-04-25T14:59:46.306741+0000","last_became_peered":"2024-04-25T14:59:46.306741+0000","last_unstale":"2024-04-25T15:01:01.487719+0000","last_undegraded":"2024-04-25T15:01:01.487719+0000","last_fullsized":"2024-04-25T15:01:01.487719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:34.407272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.501434+0000","last_change":"2024-04-25T14:59:46.306390+0000","last_active":"2024-04-25T15:01:01.501434+0000","last_peered":"2024-04-25T15:01:01.501434+0000","last_clean":"2024-04-25T15:01:01.501434+0000","last_became_active":"2024-04-25T14:59:46.306026+0000","last_became_peered":"2024-04-25T14:59:46.306026+0000","last_unstale":"2024-04-25T15:01:01.501434+0000","last_undegraded":"2024-04-25T15:01:01.501434+0000","last_fullsized":"2024-04-25T15:01:01.501434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:59.340382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.479303+0000","last_change":"2024-04-25T14:59:46.305840+0000","last_active":"2024-04-25T15:01:01.479303+0000","last_peered":"2024-04-25T15:01:01.479303+0000","last_clean":"2024-04-25T15:01:01.479303+0000","last_became_active":"2024-04-25T14:59:46.304867+0000","last_became_peered":"2024-04-25T14:59:46.304867+0000","last_unstale":"2024-04-25T15:01:01.479303+0000","last_undegraded":"2024-04-25T15:01:01.479303+0000","last_fullsized":"2024-04-25T15:01:01.479303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:58.569011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.512060+0000","last_change":"2024-04-25T14:59:46.318044+0000","last_active":"2024-04-25T15:01:01.512060+0000","last_peered":"2024-04-25T15:01:01.512060+0000","last_clean":"2024-04-25T15:01:01.512060+0000","last_became_active":"2024-04-25T14:59:46.315935+0000","last_became_peered":"2024-04-25T14:59:46.315935+0000","last_unstale":"2024-04-25T15:01:01.512060+0000","last_undegraded":"2024-04-25T15:01:01.512060+0000","last_fullsized":"2024-04-25T15:01:01.512060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:42.366802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.514545+0000","last_change":"2024-04-25T14:59:46.308565+0000","last_active":"2024-04-25T15:01:01.514545+0000","last_peered":"2024-04-25T15:01:01.514545+0000","last_clean":"2024-04-25T15:01:01.514545+0000","last_became_active":"2024-04-25T14:59:46.306198+0000","last_became_peered":"2024-04-25T14:59:46.306198+0000","last_unstale":"2024-04-25T15:01:01.514545+0000","last_undegraded":"2024-04-25T15:01:01.514545+0000","last_fullsized":"2024-04-25T15:01:01.514545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:12.686162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515623+0000","last_change":"2024-04-25T14:59:46.308313+0000","last_active":"2024-04-25T15:01:01.515623+0000","last_peered":"2024-04-25T15:01:01.515623+0000","last_clean":"2024-04-25T15:01:01.515623+0000","last_became_active":"2024-04-25T14:59:46.305974+0000","last_became_peered":"2024-04-25T14:59:46.305974+0000","last_unstale":"2024-04-25T15:01:01.515623+0000","last_undegraded":"2024-04-25T15:01:01.515623+0000","last_fullsized":"2024-04-25T15:01:01.515623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.841983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.517039+0000","last_change":"2024-04-25T14:59:46.302726+0000","last_active":"2024-04-25T15:01:01.517039+0000","last_peered":"2024-04-25T15:01:01.517039+0000","last_clean":"2024-04-25T15:01:01.517039+0000","last_became_active":"2024-04-25T14:59:46.302406+0000","last_became_peered":"2024-04-25T14:59:46.302406+0000","last_unstale":"2024-04-25T15:01:01.517039+0000","last_undegraded":"2024-04-25T15:01:01.517039+0000","last_fullsized":"2024-04-25T15:01:01.517039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519763+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:01:01.519763+0000","last_peered":"2024-04-25T15:01:01.519763+0000","last_clean":"2024-04-25T15:01:01.519763+0000","last_became_active":"2024-04-25T14:59:46.311822+0000","last_became_peered":"2024-04-25T14:59:46.311822+0000","last_unstale":"2024-04-25T15:01:01.519763+0000","last_undegraded":"2024-04-25T15:01:01.519763+0000","last_fullsized":"2024-04-25T15:01:01.519763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.128537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.504569+0000","last_change":"2024-04-25T14:59:46.313816+0000","last_active":"2024-04-25T15:01:01.504569+0000","last_peered":"2024-04-25T15:01:01.504569+0000","last_clean":"2024-04-25T15:01:01.504569+0000","last_became_active":"2024-04-25T14:59:46.313462+0000","last_became_peered":"2024-04-25T14:59:46.313462+0000","last_unstale":"2024-04-25T15:01:01.504569+0000","last_undegraded":"2024-04-25T15:01:01.504569+0000","last_fullsized":"2024-04-25T15:01:01.504569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:29.402113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.503141+0000","last_change":"2024-04-25T14:59:46.317972+0000","last_active":"2024-04-25T15:01:01.503141+0000","last_peered":"2024-04-25T15:01:01.503141+0000","last_clean":"2024-04-25T15:01:01.503141+0000","last_became_active":"2024-04-25T14:59:46.317466+0000","last_became_peered":"2024-04-25T14:59:46.317466+0000","last_unstale":"2024-04-25T15:01:01.503141+0000","last_undegraded":"2024-04-25T15:01:01.503141+0000","last_fullsized":"2024-04-25T15:01:01.503141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:03.706663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.721931+0000","last_change":"2024-04-25T14:59:45.320752+0000","last_active":"2024-04-25T15:00:02.721931+0000","last_peered":"2024-04-25T15:00:02.721931+0000","last_clean":"2024-04-25T15:00:02.721931+0000","last_became_active":"2024-04-25T14:59:45.319463+0000","last_became_peered":"2024-04-25T14:59:45.319463+0000","last_unstale":"2024-04-25T15:00:02.721931+0000","last_undegraded":"2024-04-25T15:00:02.721931+0000","last_fullsized":"2024-04-25T15:00:02.721931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:52.608899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.496871+0000","last_change":"2024-04-25T14:59:46.323802+0000","last_active":"2024-04-25T15:01:01.496871+0000","last_peered":"2024-04-25T15:01:01.496871+0000","last_clean":"2024-04-25T15:01:01.496871+0000","last_became_active":"2024-04-25T14:59:46.323498+0000","last_became_peered":"2024-04-25T14:59:46.323498+0000","last_unstale":"2024-04-25T15:01:01.496871+0000","last_undegraded":"2024-04-25T15:01:01.496871+0000","last_fullsized":"2024-04-25T15:01:01.496871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:38.236936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.713579+0000","last_change":"2024-04-25T14:59:45.335156+0000","last_active":"2024-04-25T15:00:02.713579+0000","last_peered":"2024-04-25T15:00:02.713579+0000","last_clean":"2024-04-25T15:00:02.713579+0000","last_became_active":"2024-04-25T14:59:45.333589+0000","last_became_peered":"2024-04-25T14:59:45.333589+0000","last_unstale":"2024-04-25T15:00:02.713579+0000","last_undegraded":"2024-04-25T15:00:02.713579+0000","last_fullsized":"2024-04-25T15:00:02.713579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519725+0000","last_change":"2024-04-25T14:59:46.319165+0000","last_active":"2024-04-25T15:01:01.519725+0000","last_peered":"2024-04-25T15:01:01.519725+0000","last_clean":"2024-04-25T15:01:01.519725+0000","last_became_active":"2024-04-25T14:59:46.318801+0000","last_became_peered":"2024-04-25T14:59:46.318801+0000","last_unstale":"2024-04-25T15:01:01.519725+0000","last_undegraded":"2024-04-25T15:01:01.519725+0000","last_fullsized":"2024-04-25T15:01:01.519725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:01.806099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711604+0000","last_change":"2024-04-25T14:59:45.317956+0000","last_active":"2024-04-25T15:00:02.711604+0000","last_peered":"2024-04-25T15:00:02.711604+0000","last_clean":"2024-04-25T15:00:02.711604+0000","last_became_active":"2024-04-25T14:59:45.317655+0000","last_became_peered":"2024-04-25T14:59:45.317655+0000","last_unstale":"2024-04-25T15:00:02.711604+0000","last_undegraded":"2024-04-25T15:00:02.711604+0000","last_fullsized":"2024-04-25T15:00:02.711604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:59.008411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508219+0000","last_change":"2024-04-25T14:59:46.311659+0000","last_active":"2024-04-25T15:01:01.508219+0000","last_peered":"2024-04-25T15:01:01.508219+0000","last_clean":"2024-04-25T15:01:01.508219+0000","last_became_active":"2024-04-25T14:59:46.311287+0000","last_became_peered":"2024-04-25T14:59:46.311287+0000","last_unstale":"2024-04-25T15:01:01.508219+0000","last_undegraded":"2024-04-25T15:01:01.508219+0000","last_fullsized":"2024-04-25T15:01:01.508219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:29.434223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.712212+0000","last_change":"2024-04-25T14:59:45.317658+0000","last_active":"2024-04-25T15:00:02.712212+0000","last_peered":"2024-04-25T15:00:02.712212+0000","last_clean":"2024-04-25T15:00:02.712212+0000","last_became_active":"2024-04-25T14:59:45.317311+0000","last_became_peered":"2024-04-25T14:59:45.317311+0000","last_unstale":"2024-04-25T15:00:02.712212+0000","last_undegraded":"2024-04-25T15:00:02.712212+0000","last_fullsized":"2024-04-25T15:00:02.712212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.602412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.507947+0000","last_change":"2024-04-25T14:59:46.305456+0000","last_active":"2024-04-25T15:01:01.507947+0000","last_peered":"2024-04-25T15:01:01.507947+0000","last_clean":"2024-04-25T15:01:01.507947+0000","last_became_active":"2024-04-25T14:59:46.305143+0000","last_became_peered":"2024-04-25T14:59:46.305143+0000","last_unstale":"2024-04-25T15:01:01.507947+0000","last_undegraded":"2024-04-25T15:01:01.507947+0000","last_fullsized":"2024-04-25T15:01:01.507947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:03.941200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716891+0000","last_change":"2024-04-25T14:59:45.315407+0000","last_active":"2024-04-25T15:00:02.716891+0000","last_peered":"2024-04-25T15:00:02.716891+0000","last_clean":"2024-04-25T15:00:02.716891+0000","last_became_active":"2024-04-25T14:59:45.315070+0000","last_became_peered":"2024-04-25T14:59:45.315070+0000","last_unstale":"2024-04-25T15:00:02.716891+0000","last_undegraded":"2024-04-25T15:00:02.716891+0000","last_fullsized":"2024-04-25T15:00:02.716891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:32.492027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.487654+0000","last_change":"2024-04-25T14:59:46.318218+0000","last_active":"2024-04-25T15:01:01.487654+0000","last_peered":"2024-04-25T15:01:01.487654+0000","last_clean":"2024-04-25T15:01:01.487654+0000","last_became_active":"2024-04-25T14:59:46.317855+0000","last_became_peered":"2024-04-25T14:59:46.317855+0000","last_unstale":"2024-04-25T15:01:01.487654+0000","last_undegraded":"2024-04-25T15:01:01.487654+0000","last_fullsized":"2024-04-25T15:01:01.487654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:11.712691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:03.075310+0000","last_change":"2024-04-25T14:59:45.319624+0000","last_active":"2024-04-25T15:00:03.075310+0000","last_peered":"2024-04-25T15:00:03.075310+0000","last_clean":"2024-04-25T15:00:03.075310+0000","last_became_active":"2024-04-25T14:59:45.319310+0000","last_became_peered":"2024-04-25T14:59:45.319310+0000","last_unstale":"2024-04-25T15:00:03.075310+0000","last_undegraded":"2024-04-25T15:00:03.075310+0000","last_fullsized":"2024-04-25T15:00:03.075310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:40.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506358+0000","last_change":"2024-04-25T14:59:46.317382+0000","last_active":"2024-04-25T15:01:01.506358+0000","last_peered":"2024-04-25T15:01:01.506358+0000","last_clean":"2024-04-25T15:01:01.506358+0000","last_became_active":"2024-04-25T14:59:46.316993+0000","last_became_peered":"2024-04-25T14:59:46.316993+0000","last_unstale":"2024-04-25T15:01:01.506358+0000","last_undegraded":"2024-04-25T15:01:01.506358+0000","last_fullsized":"2024-04-25T15:01:01.506358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:06.618964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.984653+0000","last_change":"2024-04-25T14:59:45.322731+0000","last_active":"2024-04-25T15:00:02.984653+0000","last_peered":"2024-04-25T15:00:02.984653+0000","last_clean":"2024-04-25T15:00:02.984653+0000","last_became_active":"2024-04-25T14:59:45.322360+0000","last_became_peered":"2024-04-25T14:59:45.322360+0000","last_unstale":"2024-04-25T15:00:02.984653+0000","last_undegraded":"2024-04-25T15:00:02.984653+0000","last_fullsized":"2024-04-25T15:00:02.984653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:02.890016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.512649+0000","last_change":"2024-04-25T14:59:46.327195+0000","last_active":"2024-04-25T15:01:01.512649+0000","last_peered":"2024-04-25T15:01:01.512649+0000","last_clean":"2024-04-25T15:01:01.512649+0000","last_became_active":"2024-04-25T14:59:46.322053+0000","last_became_peered":"2024-04-25T14:59:46.322053+0000","last_unstale":"2024-04-25T15:01:01.512649+0000","last_undegraded":"2024-04-25T15:01:01.512649+0000","last_fullsized":"2024-04-25T15:01:01.512649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:35.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716372+0000","last_change":"2024-04-25T14:59:45.315688+0000","last_active":"2024-04-25T15:00:02.716372+0000","last_peered":"2024-04-25T15:00:02.716372+0000","last_clean":"2024-04-25T15:00:02.716372+0000","last_became_active":"2024-04-25T14:59:45.315074+0000","last_became_peered":"2024-04-25T14:59:45.315074+0000","last_unstale":"2024-04-25T15:00:02.716372+0000","last_undegraded":"2024-04-25T15:00:02.716372+0000","last_fullsized":"2024-04-25T15:00:02.716372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:49.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515733+0000","last_change":"2024-04-25T14:59:46.322874+0000","last_active":"2024-04-25T15:01:01.515733+0000","last_peered":"2024-04-25T15:01:01.515733+0000","last_clean":"2024-04-25T15:01:01.515733+0000","last_became_active":"2024-04-25T14:59:46.322492+0000","last_became_peered":"2024-04-25T14:59:46.322492+0000","last_unstale":"2024-04-25T15:01:01.515733+0000","last_undegraded":"2024-04-25T15:01:01.515733+0000","last_fullsized":"2024-04-25T15:01:01.515733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:21.294792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:03.076290+0000","last_change":"2024-04-25T14:59:45.322448+0000","last_active":"2024-04-25T15:00:03.076290+0000","last_peered":"2024-04-25T15:00:03.076290+0000","last_clean":"2024-04-25T15:00:03.076290+0000","last_became_active":"2024-04-25T14:59:45.322127+0000","last_became_peered":"2024-04-25T14:59:45.322127+0000","last_unstale":"2024-04-25T15:00:03.076290+0000","last_undegraded":"2024-04-25T15:00:03.076290+0000","last_fullsized":"2024-04-25T15:00:03.076290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.284668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.507322+0000","last_change":"2024-04-25T14:59:46.318371+0000","last_active":"2024-04-25T15:01:01.507322+0000","last_peered":"2024-04-25T15:01:01.507322+0000","last_clean":"2024-04-25T15:01:01.507322+0000","last_became_active":"2024-04-25T14:59:46.317819+0000","last_became_peered":"2024-04-25T14:59:46.317819+0000","last_unstale":"2024-04-25T15:01:01.507322+0000","last_undegraded":"2024-04-25T15:01:01.507322+0000","last_fullsized":"2024-04-25T15:01:01.507322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.047982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715223+0000","last_change":"2024-04-25T14:59:45.318286+0000","last_active":"2024-04-25T15:00:02.715223+0000","last_peered":"2024-04-25T15:00:02.715223+0000","last_clean":"2024-04-25T15:00:02.715223+0000","last_became_active":"2024-04-25T14:59:45.317787+0000","last_became_peered":"2024-04-25T14:59:45.317787+0000","last_unstale":"2024-04-25T15:00:02.715223+0000","last_undegraded":"2024-04-25T15:00:02.715223+0000","last_fullsized":"2024-04-25T15:00:02.715223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:06.508164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519762+0000","last_change":"2024-04-25T15:00:02.988627+0000","last_active":"2024-04-25T15:01:01.519762+0000","last_peered":"2024-04-25T15:01:01.519762+0000","last_clean":"2024-04-25T15:01:01.519762+0000","last_became_active":"2024-04-25T15:00:02.987379+0000","last_became_peered":"2024-04-25T15:00:02.987379+0000","last_unstale":"2024-04-25T15:01:01.519762+0000","last_undegraded":"2024-04-25T15:01:01.519762+0000","last_fullsized":"2024-04-25T15:01:01.519762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:38.564177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.717001+0000","last_change":"2024-04-25T14:59:45.327951+0000","last_active":"2024-04-25T15:00:02.717001+0000","last_peered":"2024-04-25T15:00:02.717001+0000","last_clean":"2024-04-25T15:00:02.717001+0000","last_became_active":"2024-04-25T14:59:45.327660+0000","last_became_peered":"2024-04-25T14:59:45.327660+0000","last_unstale":"2024-04-25T15:00:02.717001+0000","last_undegraded":"2024-04-25T15:00:02.717001+0000","last_fullsized":"2024-04-25T15:00:02.717001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:29.862798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506904+0000","last_change":"2024-04-25T14:59:46.304038+0000","last_active":"2024-04-25T15:01:01.506904+0000","last_peered":"2024-04-25T15:01:01.506904+0000","last_clean":"2024-04-25T15:01:01.506904+0000","last_became_active":"2024-04-25T14:59:46.303713+0000","last_became_peered":"2024-04-25T14:59:46.303713+0000","last_unstale":"2024-04-25T15:01:01.506904+0000","last_undegraded":"2024-04-25T15:01:01.506904+0000","last_fullsized":"2024-04-25T15:01:01.506904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:30.476591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":210,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:05.811437+0000","last_change":"2024-04-25T14:59:45.315738+0000","last_active":"2024-04-25T15:01:05.811437+0000","last_peered":"2024-04-25T15:01:05.811437+0000","last_clean":"2024-04-25T15:01:05.811437+0000","last_became_active":"2024-04-25T14:59:45.315423+0000","last_became_peered":"2024-04-25T14:59:45.315423+0000","last_unstale":"2024-04-25T15:01:05.811437+0000","last_undegraded":"2024-04-25T15:01:05.811437+0000","last_fullsized":"2024-04-25T15:01:05.811437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:33.135348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":162,"num_read_kb":140,"num_write":29,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.488835+0000","last_change":"2024-04-25T14:59:46.308085+0000","last_active":"2024-04-25T15:01:01.488835+0000","last_peered":"2024-04-25T15:01:01.488835+0000","last_clean":"2024-04-25T15:01:01.488835+0000","last_became_active":"2024-04-25T14:59:46.307739+0000","last_became_peered":"2024-04-25T14:59:46.307739+0000","last_unstale":"2024-04-25T15:01:01.488835+0000","last_undegraded":"2024-04-25T15:01:01.488835+0000","last_fullsized":"2024-04-25T15:01:01.488835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:57.949147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711939+0000","last_change":"2024-04-25T14:59:45.327058+0000","last_active":"2024-04-25T15:00:02.711939+0000","last_peered":"2024-04-25T15:00:02.711939+0000","last_clean":"2024-04-25T15:00:02.711939+0000","last_became_active":"2024-04-25T14:59:45.326817+0000","last_became_peered":"2024-04-25T14:59:45.326817+0000","last_unstale":"2024-04-25T15:00:02.711939+0000","last_undegraded":"2024-04-25T15:00:02.711939+0000","last_fullsized":"2024-04-25T15:00:02.711939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:51.087966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.512080+0000","last_change":"2024-04-25T14:59:46.328901+0000","last_active":"2024-04-25T15:01:01.512080+0000","last_peered":"2024-04-25T15:01:01.512080+0000","last_clean":"2024-04-25T15:01:01.512080+0000","last_became_active":"2024-04-25T14:59:46.328496+0000","last_became_peered":"2024-04-25T14:59:46.328496+0000","last_unstale":"2024-04-25T15:01:01.512080+0000","last_undegraded":"2024-04-25T15:01:01.512080+0000","last_fullsized":"2024-04-25T15:01:01.512080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:48.149719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716526+0000","last_change":"2024-04-25T14:59:45.316673+0000","last_active":"2024-04-25T15:00:02.716526+0000","last_peered":"2024-04-25T15:00:02.716526+0000","last_clean":"2024-04-25T15:00:02.716526+0000","last_became_active":"2024-04-25T14:59:45.316280+0000","last_became_peered":"2024-04-25T14:59:45.316280+0000","last_unstale":"2024-04-25T15:00:02.716526+0000","last_undegraded":"2024-04-25T15:00:02.716526+0000","last_fullsized":"2024-04-25T15:00:02.716526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:51.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508449+0000","last_change":"2024-04-25T14:59:46.323821+0000","last_active":"2024-04-25T15:01:01.508449+0000","last_peered":"2024-04-25T15:01:01.508449+0000","last_clean":"2024-04-25T15:01:01.508449+0000","last_became_active":"2024-04-25T14:59:46.323434+0000","last_became_peered":"2024-04-25T14:59:46.323434+0000","last_unstale":"2024-04-25T15:01:01.508449+0000","last_undegraded":"2024-04-25T15:01:01.508449+0000","last_fullsized":"2024-04-25T15:01:01.508449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:20.983270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.955733+0000","last_change":"2024-04-25T14:59:45.317953+0000","last_active":"2024-04-25T15:00:02.955733+0000","last_peered":"2024-04-25T15:00:02.955733+0000","last_clean":"2024-04-25T15:00:02.955733+0000","last_became_active":"2024-04-25T14:59:45.317617+0000","last_became_peered":"2024-04-25T14:59:45.317617+0000","last_unstale":"2024-04-25T15:00:02.955733+0000","last_undegraded":"2024-04-25T15:00:02.955733+0000","last_fullsized":"2024-04-25T15:00:02.955733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:44.235492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.500574+0000","last_change":"2024-04-25T14:59:46.305929+0000","last_active":"2024-04-25T15:01:01.500574+0000","last_peered":"2024-04-25T15:01:01.500574+0000","last_clean":"2024-04-25T15:01:01.500574+0000","last_became_active":"2024-04-25T14:59:46.305604+0000","last_became_peered":"2024-04-25T14:59:46.305604+0000","last_unstale":"2024-04-25T15:01:01.500574+0000","last_undegraded":"2024-04-25T15:01:01.500574+0000","last_fullsized":"2024-04-25T15:01:01.500574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:37.340009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.731349+0000","last_change":"2024-04-25T15:00:02.731349+0000","last_active":"2024-04-25T15:00:02.731349+0000","last_peered":"2024-04-25T15:00:02.731349+0000","last_clean":"2024-04-25T15:00:02.731349+0000","last_became_active":"2024-04-25T15:00:02.723113+0000","last_became_peered":"2024-04-25T15:00:02.723113+0000","last_unstale":"2024-04-25T15:00:02.731349+0000","last_undegraded":"2024-04-25T15:00:02.731349+0000","last_fullsized":"2024-04-25T15:00:02.731349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:35.964461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.496783+0000","last_change":"2024-04-25T15:00:03.077923+0000","last_active":"2024-04-25T15:01:01.496783+0000","last_peered":"2024-04-25T15:01:01.496783+0000","last_clean":"2024-04-25T15:01:01.496783+0000","last_became_active":"2024-04-25T15:00:03.076709+0000","last_became_peered":"2024-04-25T15:00:03.076709+0000","last_unstale":"2024-04-25T15:01:01.496783+0000","last_undegraded":"2024-04-25T15:01:01.496783+0000","last_fullsized":"2024-04-25T15:01:01.496783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:22.427477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711970+0000","last_change":"2024-04-25T14:59:45.318291+0000","last_active":"2024-04-25T15:00:02.711970+0000","last_peered":"2024-04-25T15:00:02.711970+0000","last_clean":"2024-04-25T15:00:02.711970+0000","last_became_active":"2024-04-25T14:59:45.317590+0000","last_became_peered":"2024-04-25T14:59:45.317590+0000","last_unstale":"2024-04-25T15:00:02.711970+0000","last_undegraded":"2024-04-25T15:00:02.711970+0000","last_fullsized":"2024-04-25T15:00:02.711970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.488749+0000","last_change":"2024-04-25T14:59:46.303503+0000","last_active":"2024-04-25T15:01:01.488749+0000","last_peered":"2024-04-25T15:01:01.488749+0000","last_clean":"2024-04-25T15:01:01.488749+0000","last_became_active":"2024-04-25T14:59:46.303127+0000","last_became_peered":"2024-04-25T14:59:46.303127+0000","last_unstale":"2024-04-25T15:01:01.488749+0000","last_undegraded":"2024-04-25T15:01:01.488749+0000","last_fullsized":"2024-04-25T15:01:01.488749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:46.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.721968+0000","last_change":"2024-04-25T14:59:45.330010+0000","last_active":"2024-04-25T15:00:02.721968+0000","last_peered":"2024-04-25T15:00:02.721968+0000","last_clean":"2024-04-25T15:00:02.721968+0000","last_became_active":"2024-04-25T14:59:45.329225+0000","last_became_peered":"2024-04-25T14:59:45.329225+0000","last_unstale":"2024-04-25T15:00:02.721968+0000","last_undegraded":"2024-04-25T15:00:02.721968+0000","last_fullsized":"2024-04-25T15:00:02.721968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:01.623458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.504077+0000","last_change":"2024-04-25T14:59:46.320003+0000","last_active":"2024-04-25T15:01:01.504077+0000","last_peered":"2024-04-25T15:01:01.504077+0000","last_clean":"2024-04-25T15:01:01.504077+0000","last_became_active":"2024-04-25T14:59:46.319600+0000","last_became_peered":"2024-04-25T14:59:46.319600+0000","last_unstale":"2024-04-25T15:01:01.504077+0000","last_undegraded":"2024-04-25T15:01:01.504077+0000","last_fullsized":"2024-04-25T15:01:01.504077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.916018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715203+0000","last_change":"2024-04-25T14:59:45.325233+0000","last_active":"2024-04-25T15:00:02.715203+0000","last_peered":"2024-04-25T15:00:02.715203+0000","last_clean":"2024-04-25T15:00:02.715203+0000","last_became_active":"2024-04-25T14:59:45.324826+0000","last_became_peered":"2024-04-25T14:59:45.324826+0000","last_unstale":"2024-04-25T15:00:02.715203+0000","last_undegraded":"2024-04-25T15:00:02.715203+0000","last_fullsized":"2024-04-25T15:00:02.715203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.422559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.513615+0000","last_change":"2024-04-25T14:59:46.314552+0000","last_active":"2024-04-25T15:01:01.513615+0000","last_peered":"2024-04-25T15:01:01.513615+0000","last_clean":"2024-04-25T15:01:01.513615+0000","last_became_active":"2024-04-25T14:59:46.314159+0000","last_became_peered":"2024-04-25T14:59:46.314159+0000","last_unstale":"2024-04-25T15:01:01.513615+0000","last_undegraded":"2024-04-25T15:01:01.513615+0000","last_fullsized":"2024-04-25T15:01:01.513615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:06.731960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.491375+0000","last_change":"2024-04-25T14:59:46.303354+0000","last_active":"2024-04-25T15:01:01.491375+0000","last_peered":"2024-04-25T15:01:01.491375+0000","last_clean":"2024-04-25T15:01:01.491375+0000","last_became_active":"2024-04-25T14:59:46.302869+0000","last_became_peered":"2024-04-25T14:59:46.302869+0000","last_unstale":"2024-04-25T15:01:01.491375+0000","last_undegraded":"2024-04-25T15:01:01.491375+0000","last_fullsized":"2024-04-25T15:01:01.491375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.796902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.984556+0000","last_change":"2024-04-25T14:59:45.336686+0000","last_active":"2024-04-25T15:00:02.984556+0000","last_peered":"2024-04-25T15:00:02.984556+0000","last_clean":"2024-04-25T15:00:02.984556+0000","last_became_active":"2024-04-25T14:59:45.335828+0000","last_became_peered":"2024-04-25T14:59:45.335828+0000","last_unstale":"2024-04-25T15:00:02.984556+0000","last_undegraded":"2024-04-25T15:00:02.984556+0000","last_fullsized":"2024-04-25T15:00:02.984556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.676117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.509342+0000","last_change":"2024-04-25T14:59:46.319541+0000","last_active":"2024-04-25T15:01:01.509342+0000","last_peered":"2024-04-25T15:01:01.509342+0000","last_clean":"2024-04-25T15:01:01.509342+0000","last_became_active":"2024-04-25T14:59:46.317922+0000","last_became_peered":"2024-04-25T14:59:46.317922+0000","last_unstale":"2024-04-25T15:01:01.509342+0000","last_undegraded":"2024-04-25T15:01:01.509342+0000","last_fullsized":"2024-04-25T15:01:01.509342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.454075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.955533+0000","last_change":"2024-04-25T14:59:45.313817+0000","last_active":"2024-04-25T15:00:02.955533+0000","last_peered":"2024-04-25T15:00:02.955533+0000","last_clean":"2024-04-25T15:00:02.955533+0000","last_became_active":"2024-04-25T14:59:45.313507+0000","last_became_peered":"2024-04-25T14:59:45.313507+0000","last_unstale":"2024-04-25T15:00:02.955533+0000","last_undegraded":"2024-04-25T15:00:02.955533+0000","last_fullsized":"2024-04-25T15:00:02.955533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:18.126351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715042+0000","last_change":"2024-04-25T14:58:29.935018+0000","last_active":"2024-04-25T15:00:02.715042+0000","last_peered":"2024-04-25T15:00:02.715042+0000","last_clean":"2024-04-25T15:00:02.715042+0000","last_became_active":"2024-04-25T14:58:29.934454+0000","last_became_peered":"2024-04-25T14:58:29.934454+0000","last_unstale":"2024-04-25T15:00:02.715042+0000","last_undegraded":"2024-04-25T15:00:02.715042+0000","last_fullsized":"2024-04-25T15:00:02.715042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_clean_scrub_stamp":"2024-04-25T14:54:46.455802+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:09.308783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.496987+0000","last_change":"2024-04-25T14:59:46.319680+0000","last_active":"2024-04-25T15:01:01.496987+0000","last_peered":"2024-04-25T15:01:01.496987+0000","last_clean":"2024-04-25T15:01:01.496987+0000","last_became_active":"2024-04-25T14:59:46.319225+0000","last_became_peered":"2024-04-25T14:59:46.319225+0000","last_unstale":"2024-04-25T15:01:01.496987+0000","last_undegraded":"2024-04-25T15:01:01.496987+0000","last_fullsized":"2024-04-25T15:01:01.496987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:34.629244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715112+0000","last_change":"2024-04-25T14:59:45.318087+0000","last_active":"2024-04-25T15:00:02.715112+0000","last_peered":"2024-04-25T15:00:02.715112+0000","last_clean":"2024-04-25T15:00:02.715112+0000","last_became_active":"2024-04-25T14:59:45.317745+0000","last_became_peered":"2024-04-25T14:59:45.317745+0000","last_unstale":"2024-04-25T15:00:02.715112+0000","last_undegraded":"2024-04-25T15:00:02.715112+0000","last_fullsized":"2024-04-25T15:00:02.715112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:17.202201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711484+0000","last_change":"2024-04-25T14:59:45.332358+0000","last_active":"2024-04-25T15:00:02.711484+0000","last_peered":"2024-04-25T15:00:02.711484+0000","last_clean":"2024-04-25T15:00:02.711484+0000","last_became_active":"2024-04-25T14:59:45.332089+0000","last_became_peered":"2024-04-25T14:59:45.332089+0000","last_unstale":"2024-04-25T15:00:02.711484+0000","last_undegraded":"2024-04-25T15:00:02.711484+0000","last_fullsized":"2024-04-25T15:00:02.711484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:50.937548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.505595+0000","last_change":"2024-04-25T14:59:46.320945+0000","last_active":"2024-04-25T15:01:01.505595+0000","last_peered":"2024-04-25T15:01:01.505595+0000","last_clean":"2024-04-25T15:01:01.505595+0000","last_became_active":"2024-04-25T14:59:46.320602+0000","last_became_peered":"2024-04-25T14:59:46.320602+0000","last_unstale":"2024-04-25T15:01:01.505595+0000","last_undegraded":"2024-04-25T15:01:01.505595+0000","last_fullsized":"2024-04-25T15:01:01.505595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:32.282647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.724371+0000","last_change":"2024-04-25T14:59:45.334325+0000","last_active":"2024-04-25T15:00:02.724371+0000","last_peered":"2024-04-25T15:00:02.724371+0000","last_clean":"2024-04-25T15:00:02.724371+0000","last_became_active":"2024-04-25T14:59:45.333969+0000","last_became_peered":"2024-04-25T14:59:45.333969+0000","last_unstale":"2024-04-25T15:00:02.724371+0000","last_undegraded":"2024-04-25T15:00:02.724371+0000","last_fullsized":"2024-04-25T15:00:02.724371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:02.694267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.489178+0000","last_change":"2024-04-25T15:00:02.726558+0000","last_active":"2024-04-25T15:01:01.489178+0000","last_peered":"2024-04-25T15:01:01.489178+0000","last_clean":"2024-04-25T15:01:01.489178+0000","last_became_active":"2024-04-25T15:00:02.725467+0000","last_became_peered":"2024-04-25T15:00:02.725467+0000","last_unstale":"2024-04-25T15:01:01.489178+0000","last_undegraded":"2024-04-25T15:01:01.489178+0000","last_fullsized":"2024-04-25T15:01:01.489178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.508211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.955688+0000","last_change":"2024-04-25T14:59:45.325553+0000","last_active":"2024-04-25T15:00:02.955688+0000","last_peered":"2024-04-25T15:00:02.955688+0000","last_clean":"2024-04-25T15:00:02.955688+0000","last_became_active":"2024-04-25T14:59:45.325201+0000","last_became_peered":"2024-04-25T14:59:45.325201+0000","last_unstale":"2024-04-25T15:00:02.955688+0000","last_undegraded":"2024-04-25T15:00:02.955688+0000","last_fullsized":"2024-04-25T15:00:02.955688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:05.068269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.516873+0000","last_change":"2024-04-25T14:59:46.318252+0000","last_active":"2024-04-25T15:01:01.516873+0000","last_peered":"2024-04-25T15:01:01.516873+0000","last_clean":"2024-04-25T15:01:01.516873+0000","last_became_active":"2024-04-25T14:59:46.317601+0000","last_became_peered":"2024-04-25T14:59:46.317601+0000","last_unstale":"2024-04-25T15:01:01.516873+0000","last_undegraded":"2024-04-25T15:01:01.516873+0000","last_fullsized":"2024-04-25T15:01:01.516873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:33.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:03.074978+0000","last_change":"2024-04-25T14:59:45.329430+0000","last_active":"2024-04-25T15:00:03.074978+0000","last_peered":"2024-04-25T15:00:03.074978+0000","last_clean":"2024-04-25T15:00:03.074978+0000","last_became_active":"2024-04-25T14:59:45.329140+0000","last_became_peered":"2024-04-25T14:59:45.329140+0000","last_unstale":"2024-04-25T15:00:03.074978+0000","last_undegraded":"2024-04-25T15:00:03.074978+0000","last_fullsized":"2024-04-25T15:00:03.074978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:49.114820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.459063+0000","last_change":"2024-04-25T14:59:46.299821+0000","last_active":"2024-04-25T15:01:01.459063+0000","last_peered":"2024-04-25T15:01:01.459063+0000","last_clean":"2024-04-25T15:01:01.459063+0000","last_became_active":"2024-04-25T14:59:46.299511+0000","last_became_peered":"2024-04-25T14:59:46.299511+0000","last_unstale":"2024-04-25T15:01:01.459063+0000","last_undegraded":"2024-04-25T15:01:01.459063+0000","last_fullsized":"2024-04-25T15:01:01.459063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:23.409333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711810+0000","last_change":"2024-04-25T14:59:45.331464+0000","last_active":"2024-04-25T15:00:02.711810+0000","last_peered":"2024-04-25T15:00:02.711810+0000","last_clean":"2024-04-25T15:00:02.711810+0000","last_became_active":"2024-04-25T14:59:45.327879+0000","last_became_peered":"2024-04-25T14:59:45.327879+0000","last_unstale":"2024-04-25T15:00:02.711810+0000","last_undegraded":"2024-04-25T15:00:02.711810+0000","last_fullsized":"2024-04-25T15:00:02.711810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:50.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.518028+0000","last_change":"2024-04-25T14:59:46.310621+0000","last_active":"2024-04-25T15:01:01.518028+0000","last_peered":"2024-04-25T15:01:01.518028+0000","last_clean":"2024-04-25T15:01:01.518028+0000","last_became_active":"2024-04-25T14:59:46.310292+0000","last_became_peered":"2024-04-25T14:59:46.310292+0000","last_unstale":"2024-04-25T15:01:01.518028+0000","last_undegraded":"2024-04-25T15:01:01.518028+0000","last_fullsized":"2024-04-25T15:01:01.518028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:20.035121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.713741+0000","last_change":"2024-04-25T15:00:02.713741+0000","last_active":"2024-04-25T15:00:02.713741+0000","last_peered":"2024-04-25T15:00:02.713741+0000","last_clean":"2024-04-25T15:00:02.713741+0000","last_became_active":"2024-04-25T15:00:02.713336+0000","last_became_peered":"2024-04-25T15:00:02.713336+0000","last_unstale":"2024-04-25T15:00:02.713741+0000","last_undegraded":"2024-04-25T15:00:02.713741+0000","last_fullsized":"2024-04-25T15:00:02.713741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:37.967019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515442+0000","last_change":"2024-04-25T14:59:46.326568+0000","last_active":"2024-04-25T15:01:01.515442+0000","last_peered":"2024-04-25T15:01:01.515442+0000","last_clean":"2024-04-25T15:01:01.515442+0000","last_became_active":"2024-04-25T14:59:46.324019+0000","last_became_peered":"2024-04-25T14:59:46.324019+0000","last_unstale":"2024-04-25T15:01:01.515442+0000","last_undegraded":"2024-04-25T15:01:01.515442+0000","last_fullsized":"2024-04-25T15:01:01.515442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711345+0000","last_change":"2024-04-25T14:59:45.316935+0000","last_active":"2024-04-25T15:00:02.711345+0000","last_peered":"2024-04-25T15:00:02.711345+0000","last_clean":"2024-04-25T15:00:02.711345+0000","last_became_active":"2024-04-25T14:59:45.316604+0000","last_became_peered":"2024-04-25T14:59:45.316604+0000","last_unstale":"2024-04-25T15:00:02.711345+0000","last_undegraded":"2024-04-25T15:00:02.711345+0000","last_fullsized":"2024-04-25T15:00:02.711345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:45.699926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506260+0000","last_change":"2024-04-25T14:59:46.316195+0000","last_active":"2024-04-25T15:01:01.506260+0000","last_peered":"2024-04-25T15:01:01.506260+0000","last_clean":"2024-04-25T15:01:01.506260+0000","last_became_active":"2024-04-25T14:59:46.315809+0000","last_became_peered":"2024-04-25T14:59:46.315809+0000","last_unstale":"2024-04-25T15:01:01.506260+0000","last_undegraded":"2024-04-25T15:01:01.506260+0000","last_fullsized":"2024-04-25T15:01:01.506260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:34.440058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.712878+0000","last_change":"2024-04-25T14:59:45.325137+0000","last_active":"2024-04-25T15:00:02.712878+0000","last_peered":"2024-04-25T15:00:02.712878+0000","last_clean":"2024-04-25T15:00:02.712878+0000","last_became_active":"2024-04-25T14:59:45.324769+0000","last_became_peered":"2024-04-25T14:59:45.324769+0000","last_unstale":"2024-04-25T15:00:02.712878+0000","last_undegraded":"2024-04-25T15:00:02.712878+0000","last_fullsized":"2024-04-25T15:00:02.712878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:02.778853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.498539+0000","last_change":"2024-04-25T14:59:46.304099+0000","last_active":"2024-04-25T15:01:01.498539+0000","last_peered":"2024-04-25T15:01:01.498539+0000","last_clean":"2024-04-25T15:01:01.498539+0000","last_became_active":"2024-04-25T14:59:46.303774+0000","last_became_peered":"2024-04-25T14:59:46.303774+0000","last_unstale":"2024-04-25T15:01:01.498539+0000","last_undegraded":"2024-04-25T15:01:01.498539+0000","last_fullsized":"2024-04-25T15:01:01.498539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.308212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716806+0000","last_change":"2024-04-25T14:59:45.322324+0000","last_active":"2024-04-25T15:00:02.716806+0000","last_peered":"2024-04-25T15:00:02.716806+0000","last_clean":"2024-04-25T15:00:02.716806+0000","last_became_active":"2024-04-25T14:59:45.322031+0000","last_became_peered":"2024-04-25T14:59:45.322031+0000","last_unstale":"2024-04-25T15:00:02.716806+0000","last_undegraded":"2024-04-25T15:00:02.716806+0000","last_fullsized":"2024-04-25T15:00:02.716806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.424928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.495819+0000","last_change":"2024-04-25T15:00:02.730051+0000","last_active":"2024-04-25T15:01:01.495819+0000","last_peered":"2024-04-25T15:01:01.495819+0000","last_clean":"2024-04-25T15:01:01.495819+0000","last_became_active":"2024-04-25T15:00:02.725610+0000","last_became_peered":"2024-04-25T15:00:02.725610+0000","last_unstale":"2024-04-25T15:01:01.495819+0000","last_undegraded":"2024-04-25T15:01:01.495819+0000","last_fullsized":"2024-04-25T15:01:01.495819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:04.598366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.985136+0000","last_change":"2024-04-25T14:59:45.335502+0000","last_active":"2024-04-25T15:00:02.985136+0000","last_peered":"2024-04-25T15:00:02.985136+0000","last_clean":"2024-04-25T15:00:02.985136+0000","last_became_active":"2024-04-25T14:59:45.335018+0000","last_became_peered":"2024-04-25T14:59:45.335018+0000","last_unstale":"2024-04-25T15:00:02.985136+0000","last_undegraded":"2024-04-25T15:00:02.985136+0000","last_fullsized":"2024-04-25T15:00:02.985136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:11.918001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.501535+0000","last_change":"2024-04-25T14:59:46.304766+0000","last_active":"2024-04-25T15:01:01.501535+0000","last_peered":"2024-04-25T15:01:01.501535+0000","last_clean":"2024-04-25T15:01:01.501535+0000","last_became_active":"2024-04-25T14:59:46.304413+0000","last_became_peered":"2024-04-25T14:59:46.304413+0000","last_unstale":"2024-04-25T15:01:01.501535+0000","last_undegraded":"2024-04-25T15:01:01.501535+0000","last_fullsized":"2024-04-25T15:01:01.501535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.489458+0000","last_change":"2024-04-25T14:59:46.305285+0000","last_active":"2024-04-25T15:01:01.489458+0000","last_peered":"2024-04-25T15:01:01.489458+0000","last_clean":"2024-04-25T15:01:01.489458+0000","last_became_active":"2024-04-25T14:59:46.304934+0000","last_became_peered":"2024-04-25T14:59:46.304934+0000","last_unstale":"2024-04-25T15:01:01.489458+0000","last_undegraded":"2024-04-25T15:01:01.489458+0000","last_fullsized":"2024-04-25T15:01:01.489458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:44.892921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.497899+0000","last_change":"2024-04-25T15:00:02.718240+0000","last_active":"2024-04-25T15:01:01.497899+0000","last_peered":"2024-04-25T15:01:01.497899+0000","last_clean":"2024-04-25T15:01:01.497899+0000","last_became_active":"2024-04-25T15:00:02.717792+0000","last_became_peered":"2024-04-25T15:00:02.717792+0000","last_unstale":"2024-04-25T15:01:01.497899+0000","last_undegraded":"2024-04-25T15:01:01.497899+0000","last_fullsized":"2024-04-25T15:01:01.497899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:49.532532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.490203+0000","last_change":"2024-04-25T14:59:46.319694+0000","last_active":"2024-04-25T15:01:01.490203+0000","last_peered":"2024-04-25T15:01:01.490203+0000","last_clean":"2024-04-25T15:01:01.490203+0000","last_became_active":"2024-04-25T14:59:46.319417+0000","last_became_peered":"2024-04-25T14:59:46.319417+0000","last_unstale":"2024-04-25T15:01:01.490203+0000","last_undegraded":"2024-04-25T15:01:01.490203+0000","last_fullsized":"2024-04-25T15:01:01.490203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.177498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.513722+0000","last_change":"2024-04-25T15:00:02.986849+0000","last_active":"2024-04-25T15:01:01.513722+0000","last_peered":"2024-04-25T15:01:01.513722+0000","last_clean":"2024-04-25T15:01:01.513722+0000","last_became_active":"2024-04-25T15:00:02.986345+0000","last_became_peered":"2024-04-25T15:00:02.986345+0000","last_unstale":"2024-04-25T15:01:01.513722+0000","last_undegraded":"2024-04-25T15:01:01.513722+0000","last_fullsized":"2024-04-25T15:01:01.513722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:22.707469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.517441+0000","last_change":"2024-04-25T14:59:46.319167+0000","last_active":"2024-04-25T15:01:01.517441+0000","last_peered":"2024-04-25T15:01:01.517441+0000","last_clean":"2024-04-25T15:01:01.517441+0000","last_became_active":"2024-04-25T14:59:46.318804+0000","last_became_peered":"2024-04-25T14:59:46.318804+0000","last_unstale":"2024-04-25T15:01:01.517441+0000","last_undegraded":"2024-04-25T15:01:01.517441+0000","last_fullsized":"2024-04-25T15:01:01.517441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:34.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.488823+0000","last_change":"2024-04-25T14:59:46.304554+0000","last_active":"2024-04-25T15:01:01.488823+0000","last_peered":"2024-04-25T15:01:01.488823+0000","last_clean":"2024-04-25T15:01:01.488823+0000","last_became_active":"2024-04-25T14:59:46.304062+0000","last_became_peered":"2024-04-25T14:59:46.304062+0000","last_unstale":"2024-04-25T15:01:01.488823+0000","last_undegraded":"2024-04-25T15:01:01.488823+0000","last_fullsized":"2024-04-25T15:01:01.488823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.481095+0000","last_change":"2024-04-25T14:59:46.299104+0000","last_active":"2024-04-25T15:01:01.481095+0000","last_peered":"2024-04-25T15:01:01.481095+0000","last_clean":"2024-04-25T15:01:01.481095+0000","last_became_active":"2024-04-25T14:59:46.298765+0000","last_became_peered":"2024-04-25T14:59:46.298765+0000","last_unstale":"2024-04-25T15:01:01.481095+0000","last_undegraded":"2024-04-25T15:01:01.481095+0000","last_fullsized":"2024-04-25T15:01:01.481095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.494298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.502770+0000","last_change":"2024-04-25T14:59:46.328885+0000","last_active":"2024-04-25T15:01:01.502770+0000","last_peered":"2024-04-25T15:01:01.502770+0000","last_clean":"2024-04-25T15:01:01.502770+0000","last_became_active":"2024-04-25T14:59:46.328375+0000","last_became_peered":"2024-04-25T14:59:46.328375+0000","last_unstale":"2024-04-25T15:01:01.502770+0000","last_undegraded":"2024-04-25T15:01:01.502770+0000","last_fullsized":"2024-04-25T15:01:01.502770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:08.959045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.505955+0000","last_change":"2024-04-25T14:59:46.317403+0000","last_active":"2024-04-25T15:01:01.505955+0000","last_peered":"2024-04-25T15:01:01.505955+0000","last_clean":"2024-04-25T15:01:01.505955+0000","last_became_active":"2024-04-25T14:59:46.317100+0000","last_became_peered":"2024-04-25T14:59:46.317100+0000","last_unstale":"2024-04-25T15:01:01.505955+0000","last_undegraded":"2024-04-25T15:01:01.505955+0000","last_fullsized":"2024-04-25T15:01:01.505955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:23.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.509148+0000","last_change":"2024-04-25T14:59:46.327147+0000","last_active":"2024-04-25T15:01:01.509148+0000","last_peered":"2024-04-25T15:01:01.509148+0000","last_clean":"2024-04-25T15:01:01.509148+0000","last_became_active":"2024-04-25T14:59:46.321857+0000","last_became_peered":"2024-04-25T14:59:46.321857+0000","last_unstale":"2024-04-25T15:01:01.509148+0000","last_undegraded":"2024-04-25T15:01:01.509148+0000","last_fullsized":"2024-04-25T15:01:01.509148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:46.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508618+0000","last_change":"2024-04-25T14:59:46.310841+0000","last_active":"2024-04-25T15:01:01.508618+0000","last_peered":"2024-04-25T15:01:01.508618+0000","last_clean":"2024-04-25T15:01:01.508618+0000","last_became_active":"2024-04-25T14:59:46.310446+0000","last_became_peered":"2024-04-25T14:59:46.310446+0000","last_unstale":"2024-04-25T15:01:01.508618+0000","last_undegraded":"2024-04-25T15:01:01.508618+0000","last_fullsized":"2024-04-25T15:01:01.508618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.594784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.516706+0000","last_change":"2024-04-25T14:59:46.320715+0000","last_active":"2024-04-25T15:01:01.516706+0000","last_peered":"2024-04-25T15:01:01.516706+0000","last_clean":"2024-04-25T15:01:01.516706+0000","last_became_active":"2024-04-25T14:59:46.320238+0000","last_became_peered":"2024-04-25T14:59:46.320238+0000","last_unstale":"2024-04-25T15:01:01.516706+0000","last_undegraded":"2024-04-25T15:01:01.516706+0000","last_fullsized":"2024-04-25T15:01:01.516706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:55.387814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.484458+0000","last_change":"2024-04-25T14:59:46.303772+0000","last_active":"2024-04-25T15:01:01.484458+0000","last_peered":"2024-04-25T15:01:01.484458+0000","last_clean":"2024-04-25T15:01:01.484458+0000","last_became_active":"2024-04-25T14:59:46.303435+0000","last_became_peered":"2024-04-25T14:59:46.303435+0000","last_unstale":"2024-04-25T15:01:01.484458+0000","last_undegraded":"2024-04-25T15:01:01.484458+0000","last_fullsized":"2024-04-25T15:01:01.484458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:06.692979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62093,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":168,"num_read_kb":146,"num_write":80,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":186279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":458753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1110727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":459452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":459357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1049820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":458870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":520048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":458843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":520082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":458974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":458775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1049420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:01:14.264 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:01:14.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:14 smithi138 ceph-mon[32875]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:01:14.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:14 smithi138 ceph-mon[32875]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:14.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:14 smithi137 ceph-mon[32439]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:01:14.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:14 smithi137 ceph-mon[32439]: pgmap v354: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:01:14.758 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:16.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:16 smithi094 ceph-mon[27862]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 -- 172.21.15.94:0/4233219647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 msgr2=0x7f4e9010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 --2- 172.21.15.94:0/4233219647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 0x7f4e9010e7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800b0a0 tx=0x7f4e8800b3b0 comp rx=0 tx=0).stop 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 -- 172.21.15.94:0/4233219647 shutdown_connections 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 --2- 172.21.15.94:0/4233219647 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 0x7f4e9010e7f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 --2- 172.21.15.94:0/4233219647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e90107990 0x7f4e90107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 --2- 172.21.15.94:0/4233219647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90101940 0x7f4e90101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 -- 172.21.15.94:0/4233219647 >> 172.21.15.94:0/4233219647 conn(0x7f4e900788f0 msgr2=0x7f4e90078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:16.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 -- 172.21.15.94:0/4233219647 shutdown_connections 2024-04-25T15:01:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.946+0000 7f4e95800700 1 -- 172.21.15.94:0/4233219647 wait complete. 2024-04-25T15:01:16.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.947+0000 7f4e95800700 1 Processor -- start 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e95800700 1 -- start start 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e95800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e90101940 0x7f4e901990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e95800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 0x7f4e90199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e95800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90107990 0x7f4e9019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e95800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90112c50 con 0x7f4e90107990 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e95800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90112ad0 con 0x7f4e90102300 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e95800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90112dd0 con 0x7f4e90101940 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e8f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90107990 0x7f4e9019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e8ea00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e90101940 0x7f4e901990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e8f400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90107990 0x7f4e9019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51202/0 (socket says 172.21.15.94:51202) 2024-04-25T15:01:16.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e8f400700 1 -- 172.21.15.94:0/921206679 learned_addr learned my addr 172.21.15.94:0/921206679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:16.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.948+0000 7f4e8e000700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 0x7f4e90199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e8ea00700 1 -- 172.21.15.94:0/921206679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 msgr2=0x7f4e90199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e8ea00700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 0x7f4e90199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e8ea00700 1 -- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90107990 msgr2=0x7f4e9019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e8ea00700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90107990 0x7f4e9019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e8ea00700 1 -- 172.21.15.94:0/921206679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9019e0d0 con 0x7f4e90101940 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e8f400700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90107990 0x7f4e9019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e8ea00700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e90101940 0x7f4e901990c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000e9f0 tx=0x7f4e8000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e87400700 1 -- 172.21.15.94:0/921206679 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8000c9c0 con 0x7f4e90101940 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.949+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9019e3c0 con 0x7f4e90101940 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.950+0000 7f4e87400700 1 -- 172.21.15.94:0/921206679 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8000cb60 con 0x7f4e90101940 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.950+0000 7f4e87400700 1 -- 172.21.15.94:0/921206679 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80018680 con 0x7f4e90101940 2024-04-25T15:01:16.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.950+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e901a5c40 con 0x7f4e90101940 2024-04-25T15:01:16.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.951+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e90069730 con 0x7f4e90101940 2024-04-25T15:01:16.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.953+0000 7f4e87400700 1 -- 172.21.15.94:0/921206679 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4e80010430 con 0x7f4e90101940 2024-04-25T15:01:16.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.954+0000 7f4e87400700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e740a3cc0 0x7f4e740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.954+0000 7f4e87400700 1 -- 172.21.15.94:0/921206679 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4e800d0470 con 0x7f4e90101940 2024-04-25T15:01:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.954+0000 7f4e8e000700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e740a3cc0 0x7f4e740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:16.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.955+0000 7f4e8e000700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e740a3cc0 0x7f4e740a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e90078ef0 tx=0x7f4e7c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:16.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:16 smithi138 ceph-mon[32875]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T15:01:16.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:16.959+0000 7f4e87400700 1 -- 172.21.15.94:0/921206679 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e8009ce10 con 0x7f4e90101940 2024-04-25T15:01:16.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:16 smithi137 ceph-mon[32439]: pgmap v355: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 936 B/s rd, 0 B/s wr, 1 op/s 2024-04-25T15:01:17.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.176+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e90062e80 con 0x7f4e740a3cc0 2024-04-25T15:01:17.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.196+0000 7f4e87400700 1 -- 172.21.15.94:0/921206679 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258747 (secure 0 0 0) 0x7f4e90062e80 con 0x7f4e740a3cc0 2024-04-25T15:01:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e740a3cc0 msgr2=0x7f4e740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e740a3cc0 0x7f4e740a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e90078ef0 tx=0x7f4e7c009260 comp rx=0 tx=0).stop 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e90101940 msgr2=0x7f4e901990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e90101940 0x7f4e901990c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000e9f0 tx=0x7f4e8000ed00 comp rx=0 tx=0).stop 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 shutdown_connections 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4e740a3cc0 0x7f4e740a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e90107990 0x7f4e9019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e90101940 0x7f4e901990c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 --2- 172.21.15.94:0/921206679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4e90102300 0x7f4e90199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 >> 172.21.15.94:0/921206679 conn(0x7f4e900788f0 msgr2=0x7f4e900ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 shutdown_connections 2024-04-25T15:01:17.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:17.203+0000 7f4e95800700 1 -- 172.21.15.94:0/921206679 wait complete. 2024-04-25T15:01:17.209 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T15:01:18.113 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-04-25T15:01:15.529356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652695,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":314,"num_read_kb":270,"num_write":263,"num_write_kb":3026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":707,"ondisk_log_size":707,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379004,"kb_used_data":13212,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562820,"statfs":{"total":1151940427776,"available":1151552327680,"internally_reserved":0,"allocated":13529088,"data_stored":7463121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.024701"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519590+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:01:01.519590+0000","last_peered":"2024-04-25T15:01:01.519590+0000","last_clean":"2024-04-25T15:01:01.519590+0000","last_became_active":"2024-04-25T14:59:46.311856+0000","last_became_peered":"2024-04-25T14:59:46.311856+0000","last_unstale":"2024-04-25T15:01:01.519590+0000","last_undegraded":"2024-04-25T15:01:01.519590+0000","last_fullsized":"2024-04-25T15:01:01.519590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:07.469645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.514143+0000","last_change":"2024-04-25T14:59:46.310292+0000","last_active":"2024-04-25T15:01:01.514143+0000","last_peered":"2024-04-25T15:01:01.514143+0000","last_clean":"2024-04-25T15:01:01.514143+0000","last_became_active":"2024-04-25T14:59:46.309812+0000","last_became_peered":"2024-04-25T14:59:46.309812+0000","last_unstale":"2024-04-25T15:01:01.514143+0000","last_undegraded":"2024-04-25T15:01:01.514143+0000","last_fullsized":"2024-04-25T15:01:01.514143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:52.448406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515414+0000","last_change":"2024-04-25T14:59:46.300987+0000","last_active":"2024-04-25T15:01:01.515414+0000","last_peered":"2024-04-25T15:01:01.515414+0000","last_clean":"2024-04-25T15:01:01.515414+0000","last_became_active":"2024-04-25T14:59:46.299866+0000","last_became_peered":"2024-04-25T14:59:46.299866+0000","last_unstale":"2024-04-25T15:01:01.515414+0000","last_undegraded":"2024-04-25T15:01:01.515414+0000","last_fullsized":"2024-04-25T15:01:01.515414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:09.831131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.484945+0000","last_change":"2024-04-25T14:59:46.304278+0000","last_active":"2024-04-25T15:01:01.484945+0000","last_peered":"2024-04-25T15:01:01.484945+0000","last_clean":"2024-04-25T15:01:01.484945+0000","last_became_active":"2024-04-25T14:59:46.303956+0000","last_became_peered":"2024-04-25T14:59:46.303956+0000","last_unstale":"2024-04-25T15:01:01.484945+0000","last_undegraded":"2024-04-25T15:01:01.484945+0000","last_fullsized":"2024-04-25T15:01:01.484945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.752548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.503892+0000","last_change":"2024-04-25T14:59:46.323730+0000","last_active":"2024-04-25T15:01:01.503892+0000","last_peered":"2024-04-25T15:01:01.503892+0000","last_clean":"2024-04-25T15:01:01.503892+0000","last_became_active":"2024-04-25T14:59:46.323383+0000","last_became_peered":"2024-04-25T14:59:46.323383+0000","last_unstale":"2024-04-25T15:01:01.503892+0000","last_undegraded":"2024-04-25T15:01:01.503892+0000","last_fullsized":"2024-04-25T15:01:01.503892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.846392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506646+0000","last_change":"2024-04-25T14:59:46.317623+0000","last_active":"2024-04-25T15:01:01.506646+0000","last_peered":"2024-04-25T15:01:01.506646+0000","last_clean":"2024-04-25T15:01:01.506646+0000","last_became_active":"2024-04-25T14:59:46.317255+0000","last_became_peered":"2024-04-25T14:59:46.317255+0000","last_unstale":"2024-04-25T15:01:01.506646+0000","last_undegraded":"2024-04-25T15:01:01.506646+0000","last_fullsized":"2024-04-25T15:01:01.506646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:24.449615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.487029+0000","last_change":"2024-04-25T15:00:02.723289+0000","last_active":"2024-04-25T15:01:01.487029+0000","last_peered":"2024-04-25T15:01:01.487029+0000","last_clean":"2024-04-25T15:01:01.487029+0000","last_became_active":"2024-04-25T15:00:02.722968+0000","last_became_peered":"2024-04-25T15:00:02.722968+0000","last_unstale":"2024-04-25T15:01:01.487029+0000","last_undegraded":"2024-04-25T15:01:01.487029+0000","last_fullsized":"2024-04-25T15:01:01.487029+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:45.674905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508192+0000","last_change":"2024-04-25T14:59:46.327687+0000","last_active":"2024-04-25T15:01:01.508192+0000","last_peered":"2024-04-25T15:01:01.508192+0000","last_clean":"2024-04-25T15:01:01.508192+0000","last_became_active":"2024-04-25T14:59:46.325905+0000","last_became_peered":"2024-04-25T14:59:46.325905+0000","last_unstale":"2024-04-25T15:01:01.508192+0000","last_undegraded":"2024-04-25T15:01:01.508192+0000","last_fullsized":"2024-04-25T15:01:01.508192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:50.058801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.505083+0000","last_change":"2024-04-25T14:59:46.304722+0000","last_active":"2024-04-25T15:01:01.505083+0000","last_peered":"2024-04-25T15:01:01.505083+0000","last_clean":"2024-04-25T15:01:01.505083+0000","last_became_active":"2024-04-25T14:59:46.304391+0000","last_became_peered":"2024-04-25T14:59:46.304391+0000","last_unstale":"2024-04-25T15:01:01.505083+0000","last_undegraded":"2024-04-25T15:01:01.505083+0000","last_fullsized":"2024-04-25T15:01:01.505083+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:15.735670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.487719+0000","last_change":"2024-04-25T14:59:46.307796+0000","last_active":"2024-04-25T15:01:01.487719+0000","last_peered":"2024-04-25T15:01:01.487719+0000","last_clean":"2024-04-25T15:01:01.487719+0000","last_became_active":"2024-04-25T14:59:46.306741+0000","last_became_peered":"2024-04-25T14:59:46.306741+0000","last_unstale":"2024-04-25T15:01:01.487719+0000","last_undegraded":"2024-04-25T15:01:01.487719+0000","last_fullsized":"2024-04-25T15:01:01.487719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:34.407272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.501434+0000","last_change":"2024-04-25T14:59:46.306390+0000","last_active":"2024-04-25T15:01:01.501434+0000","last_peered":"2024-04-25T15:01:01.501434+0000","last_clean":"2024-04-25T15:01:01.501434+0000","last_became_active":"2024-04-25T14:59:46.306026+0000","last_became_peered":"2024-04-25T14:59:46.306026+0000","last_unstale":"2024-04-25T15:01:01.501434+0000","last_undegraded":"2024-04-25T15:01:01.501434+0000","last_fullsized":"2024-04-25T15:01:01.501434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:59.340382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.479303+0000","last_change":"2024-04-25T14:59:46.305840+0000","last_active":"2024-04-25T15:01:01.479303+0000","last_peered":"2024-04-25T15:01:01.479303+0000","last_clean":"2024-04-25T15:01:01.479303+0000","last_became_active":"2024-04-25T14:59:46.304867+0000","last_became_peered":"2024-04-25T14:59:46.304867+0000","last_unstale":"2024-04-25T15:01:01.479303+0000","last_undegraded":"2024-04-25T15:01:01.479303+0000","last_fullsized":"2024-04-25T15:01:01.479303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:58.569011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.512060+0000","last_change":"2024-04-25T14:59:46.318044+0000","last_active":"2024-04-25T15:01:01.512060+0000","last_peered":"2024-04-25T15:01:01.512060+0000","last_clean":"2024-04-25T15:01:01.512060+0000","last_became_active":"2024-04-25T14:59:46.315935+0000","last_became_peered":"2024-04-25T14:59:46.315935+0000","last_unstale":"2024-04-25T15:01:01.512060+0000","last_undegraded":"2024-04-25T15:01:01.512060+0000","last_fullsized":"2024-04-25T15:01:01.512060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:42.366802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.514545+0000","last_change":"2024-04-25T14:59:46.308565+0000","last_active":"2024-04-25T15:01:01.514545+0000","last_peered":"2024-04-25T15:01:01.514545+0000","last_clean":"2024-04-25T15:01:01.514545+0000","last_became_active":"2024-04-25T14:59:46.306198+0000","last_became_peered":"2024-04-25T14:59:46.306198+0000","last_unstale":"2024-04-25T15:01:01.514545+0000","last_undegraded":"2024-04-25T15:01:01.514545+0000","last_fullsized":"2024-04-25T15:01:01.514545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:12.686162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515623+0000","last_change":"2024-04-25T14:59:46.308313+0000","last_active":"2024-04-25T15:01:01.515623+0000","last_peered":"2024-04-25T15:01:01.515623+0000","last_clean":"2024-04-25T15:01:01.515623+0000","last_became_active":"2024-04-25T14:59:46.305974+0000","last_became_peered":"2024-04-25T14:59:46.305974+0000","last_unstale":"2024-04-25T15:01:01.515623+0000","last_undegraded":"2024-04-25T15:01:01.515623+0000","last_fullsized":"2024-04-25T15:01:01.515623+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.841983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.517039+0000","last_change":"2024-04-25T14:59:46.302726+0000","last_active":"2024-04-25T15:01:01.517039+0000","last_peered":"2024-04-25T15:01:01.517039+0000","last_clean":"2024-04-25T15:01:01.517039+0000","last_became_active":"2024-04-25T14:59:46.302406+0000","last_became_peered":"2024-04-25T14:59:46.302406+0000","last_unstale":"2024-04-25T15:01:01.517039+0000","last_undegraded":"2024-04-25T15:01:01.517039+0000","last_fullsized":"2024-04-25T15:01:01.517039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519763+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:01:01.519763+0000","last_peered":"2024-04-25T15:01:01.519763+0000","last_clean":"2024-04-25T15:01:01.519763+0000","last_became_active":"2024-04-25T14:59:46.311822+0000","last_became_peered":"2024-04-25T14:59:46.311822+0000","last_unstale":"2024-04-25T15:01:01.519763+0000","last_undegraded":"2024-04-25T15:01:01.519763+0000","last_fullsized":"2024-04-25T15:01:01.519763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.128537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.504569+0000","last_change":"2024-04-25T14:59:46.313816+0000","last_active":"2024-04-25T15:01:01.504569+0000","last_peered":"2024-04-25T15:01:01.504569+0000","last_clean":"2024-04-25T15:01:01.504569+0000","last_became_active":"2024-04-25T14:59:46.313462+0000","last_became_peered":"2024-04-25T14:59:46.313462+0000","last_unstale":"2024-04-25T15:01:01.504569+0000","last_undegraded":"2024-04-25T15:01:01.504569+0000","last_fullsized":"2024-04-25T15:01:01.504569+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:29.402113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.503141+0000","last_change":"2024-04-25T14:59:46.317972+0000","last_active":"2024-04-25T15:01:01.503141+0000","last_peered":"2024-04-25T15:01:01.503141+0000","last_clean":"2024-04-25T15:01:01.503141+0000","last_became_active":"2024-04-25T14:59:46.317466+0000","last_became_peered":"2024-04-25T14:59:46.317466+0000","last_unstale":"2024-04-25T15:01:01.503141+0000","last_undegraded":"2024-04-25T15:01:01.503141+0000","last_fullsized":"2024-04-25T15:01:01.503141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:03.706663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.721931+0000","last_change":"2024-04-25T14:59:45.320752+0000","last_active":"2024-04-25T15:00:02.721931+0000","last_peered":"2024-04-25T15:00:02.721931+0000","last_clean":"2024-04-25T15:00:02.721931+0000","last_became_active":"2024-04-25T14:59:45.319463+0000","last_became_peered":"2024-04-25T14:59:45.319463+0000","last_unstale":"2024-04-25T15:00:02.721931+0000","last_undegraded":"2024-04-25T15:00:02.721931+0000","last_fullsized":"2024-04-25T15:00:02.721931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:52.608899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.496871+0000","last_change":"2024-04-25T14:59:46.323802+0000","last_active":"2024-04-25T15:01:01.496871+0000","last_peered":"2024-04-25T15:01:01.496871+0000","last_clean":"2024-04-25T15:01:01.496871+0000","last_became_active":"2024-04-25T14:59:46.323498+0000","last_became_peered":"2024-04-25T14:59:46.323498+0000","last_unstale":"2024-04-25T15:01:01.496871+0000","last_undegraded":"2024-04-25T15:01:01.496871+0000","last_fullsized":"2024-04-25T15:01:01.496871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:38.236936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.713579+0000","last_change":"2024-04-25T14:59:45.335156+0000","last_active":"2024-04-25T15:00:02.713579+0000","last_peered":"2024-04-25T15:00:02.713579+0000","last_clean":"2024-04-25T15:00:02.713579+0000","last_became_active":"2024-04-25T14:59:45.333589+0000","last_became_peered":"2024-04-25T14:59:45.333589+0000","last_unstale":"2024-04-25T15:00:02.713579+0000","last_undegraded":"2024-04-25T15:00:02.713579+0000","last_fullsized":"2024-04-25T15:00:02.713579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519725+0000","last_change":"2024-04-25T14:59:46.319165+0000","last_active":"2024-04-25T15:01:01.519725+0000","last_peered":"2024-04-25T15:01:01.519725+0000","last_clean":"2024-04-25T15:01:01.519725+0000","last_became_active":"2024-04-25T14:59:46.318801+0000","last_became_peered":"2024-04-25T14:59:46.318801+0000","last_unstale":"2024-04-25T15:01:01.519725+0000","last_undegraded":"2024-04-25T15:01:01.519725+0000","last_fullsized":"2024-04-25T15:01:01.519725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:01.806099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711604+0000","last_change":"2024-04-25T14:59:45.317956+0000","last_active":"2024-04-25T15:00:02.711604+0000","last_peered":"2024-04-25T15:00:02.711604+0000","last_clean":"2024-04-25T15:00:02.711604+0000","last_became_active":"2024-04-25T14:59:45.317655+0000","last_became_peered":"2024-04-25T14:59:45.317655+0000","last_unstale":"2024-04-25T15:00:02.711604+0000","last_undegraded":"2024-04-25T15:00:02.711604+0000","last_fullsized":"2024-04-25T15:00:02.711604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:59.008411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508219+0000","last_change":"2024-04-25T14:59:46.311659+0000","last_active":"2024-04-25T15:01:01.508219+0000","last_peered":"2024-04-25T15:01:01.508219+0000","last_clean":"2024-04-25T15:01:01.508219+0000","last_became_active":"2024-04-25T14:59:46.311287+0000","last_became_peered":"2024-04-25T14:59:46.311287+0000","last_unstale":"2024-04-25T15:01:01.508219+0000","last_undegraded":"2024-04-25T15:01:01.508219+0000","last_fullsized":"2024-04-25T15:01:01.508219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:29.434223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.712212+0000","last_change":"2024-04-25T14:59:45.317658+0000","last_active":"2024-04-25T15:00:02.712212+0000","last_peered":"2024-04-25T15:00:02.712212+0000","last_clean":"2024-04-25T15:00:02.712212+0000","last_became_active":"2024-04-25T14:59:45.317311+0000","last_became_peered":"2024-04-25T14:59:45.317311+0000","last_unstale":"2024-04-25T15:00:02.712212+0000","last_undegraded":"2024-04-25T15:00:02.712212+0000","last_fullsized":"2024-04-25T15:00:02.712212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.602412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.507947+0000","last_change":"2024-04-25T14:59:46.305456+0000","last_active":"2024-04-25T15:01:01.507947+0000","last_peered":"2024-04-25T15:01:01.507947+0000","last_clean":"2024-04-25T15:01:01.507947+0000","last_became_active":"2024-04-25T14:59:46.305143+0000","last_became_peered":"2024-04-25T14:59:46.305143+0000","last_unstale":"2024-04-25T15:01:01.507947+0000","last_undegraded":"2024-04-25T15:01:01.507947+0000","last_fullsized":"2024-04-25T15:01:01.507947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:03.941200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716891+0000","last_change":"2024-04-25T14:59:45.315407+0000","last_active":"2024-04-25T15:00:02.716891+0000","last_peered":"2024-04-25T15:00:02.716891+0000","last_clean":"2024-04-25T15:00:02.716891+0000","last_became_active":"2024-04-25T14:59:45.315070+0000","last_became_peered":"2024-04-25T14:59:45.315070+0000","last_unstale":"2024-04-25T15:00:02.716891+0000","last_undegraded":"2024-04-25T15:00:02.716891+0000","last_fullsized":"2024-04-25T15:00:02.716891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:32.492027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.487654+0000","last_change":"2024-04-25T14:59:46.318218+0000","last_active":"2024-04-25T15:01:01.487654+0000","last_peered":"2024-04-25T15:01:01.487654+0000","last_clean":"2024-04-25T15:01:01.487654+0000","last_became_active":"2024-04-25T14:59:46.317855+0000","last_became_peered":"2024-04-25T14:59:46.317855+0000","last_unstale":"2024-04-25T15:01:01.487654+0000","last_undegraded":"2024-04-25T15:01:01.487654+0000","last_fullsized":"2024-04-25T15:01:01.487654+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:11.712691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:03.075310+0000","last_change":"2024-04-25T14:59:45.319624+0000","last_active":"2024-04-25T15:00:03.075310+0000","last_peered":"2024-04-25T15:00:03.075310+0000","last_clean":"2024-04-25T15:00:03.075310+0000","last_became_active":"2024-04-25T14:59:45.319310+0000","last_became_peered":"2024-04-25T14:59:45.319310+0000","last_unstale":"2024-04-25T15:00:03.075310+0000","last_undegraded":"2024-04-25T15:00:03.075310+0000","last_fullsized":"2024-04-25T15:00:03.075310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:40.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506358+0000","last_change":"2024-04-25T14:59:46.317382+0000","last_active":"2024-04-25T15:01:01.506358+0000","last_peered":"2024-04-25T15:01:01.506358+0000","last_clean":"2024-04-25T15:01:01.506358+0000","last_became_active":"2024-04-25T14:59:46.316993+0000","last_became_peered":"2024-04-25T14:59:46.316993+0000","last_unstale":"2024-04-25T15:01:01.506358+0000","last_undegraded":"2024-04-25T15:01:01.506358+0000","last_fullsized":"2024-04-25T15:01:01.506358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:06.618964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.984653+0000","last_change":"2024-04-25T14:59:45.322731+0000","last_active":"2024-04-25T15:00:02.984653+0000","last_peered":"2024-04-25T15:00:02.984653+0000","last_clean":"2024-04-25T15:00:02.984653+0000","last_became_active":"2024-04-25T14:59:45.322360+0000","last_became_peered":"2024-04-25T14:59:45.322360+0000","last_unstale":"2024-04-25T15:00:02.984653+0000","last_undegraded":"2024-04-25T15:00:02.984653+0000","last_fullsized":"2024-04-25T15:00:02.984653+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:02.890016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.512649+0000","last_change":"2024-04-25T14:59:46.327195+0000","last_active":"2024-04-25T15:01:01.512649+0000","last_peered":"2024-04-25T15:01:01.512649+0000","last_clean":"2024-04-25T15:01:01.512649+0000","last_became_active":"2024-04-25T14:59:46.322053+0000","last_became_peered":"2024-04-25T14:59:46.322053+0000","last_unstale":"2024-04-25T15:01:01.512649+0000","last_undegraded":"2024-04-25T15:01:01.512649+0000","last_fullsized":"2024-04-25T15:01:01.512649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:35.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716372+0000","last_change":"2024-04-25T14:59:45.315688+0000","last_active":"2024-04-25T15:00:02.716372+0000","last_peered":"2024-04-25T15:00:02.716372+0000","last_clean":"2024-04-25T15:00:02.716372+0000","last_became_active":"2024-04-25T14:59:45.315074+0000","last_became_peered":"2024-04-25T14:59:45.315074+0000","last_unstale":"2024-04-25T15:00:02.716372+0000","last_undegraded":"2024-04-25T15:00:02.716372+0000","last_fullsized":"2024-04-25T15:00:02.716372+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:49.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515733+0000","last_change":"2024-04-25T14:59:46.322874+0000","last_active":"2024-04-25T15:01:01.515733+0000","last_peered":"2024-04-25T15:01:01.515733+0000","last_clean":"2024-04-25T15:01:01.515733+0000","last_became_active":"2024-04-25T14:59:46.322492+0000","last_became_peered":"2024-04-25T14:59:46.322492+0000","last_unstale":"2024-04-25T15:01:01.515733+0000","last_undegraded":"2024-04-25T15:01:01.515733+0000","last_fullsized":"2024-04-25T15:01:01.515733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:21.294792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:03.076290+0000","last_change":"2024-04-25T14:59:45.322448+0000","last_active":"2024-04-25T15:00:03.076290+0000","last_peered":"2024-04-25T15:00:03.076290+0000","last_clean":"2024-04-25T15:00:03.076290+0000","last_became_active":"2024-04-25T14:59:45.322127+0000","last_became_peered":"2024-04-25T14:59:45.322127+0000","last_unstale":"2024-04-25T15:00:03.076290+0000","last_undegraded":"2024-04-25T15:00:03.076290+0000","last_fullsized":"2024-04-25T15:00:03.076290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.284668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.507322+0000","last_change":"2024-04-25T14:59:46.318371+0000","last_active":"2024-04-25T15:01:01.507322+0000","last_peered":"2024-04-25T15:01:01.507322+0000","last_clean":"2024-04-25T15:01:01.507322+0000","last_became_active":"2024-04-25T14:59:46.317819+0000","last_became_peered":"2024-04-25T14:59:46.317819+0000","last_unstale":"2024-04-25T15:01:01.507322+0000","last_undegraded":"2024-04-25T15:01:01.507322+0000","last_fullsized":"2024-04-25T15:01:01.507322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.047982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715223+0000","last_change":"2024-04-25T14:59:45.318286+0000","last_active":"2024-04-25T15:00:02.715223+0000","last_peered":"2024-04-25T15:00:02.715223+0000","last_clean":"2024-04-25T15:00:02.715223+0000","last_became_active":"2024-04-25T14:59:45.317787+0000","last_became_peered":"2024-04-25T14:59:45.317787+0000","last_unstale":"2024-04-25T15:00:02.715223+0000","last_undegraded":"2024-04-25T15:00:02.715223+0000","last_fullsized":"2024-04-25T15:00:02.715223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:06.508164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.519762+0000","last_change":"2024-04-25T15:00:02.988627+0000","last_active":"2024-04-25T15:01:01.519762+0000","last_peered":"2024-04-25T15:01:01.519762+0000","last_clean":"2024-04-25T15:01:01.519762+0000","last_became_active":"2024-04-25T15:00:02.987379+0000","last_became_peered":"2024-04-25T15:00:02.987379+0000","last_unstale":"2024-04-25T15:01:01.519762+0000","last_undegraded":"2024-04-25T15:01:01.519762+0000","last_fullsized":"2024-04-25T15:01:01.519762+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:38.564177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.717001+0000","last_change":"2024-04-25T14:59:45.327951+0000","last_active":"2024-04-25T15:00:02.717001+0000","last_peered":"2024-04-25T15:00:02.717001+0000","last_clean":"2024-04-25T15:00:02.717001+0000","last_became_active":"2024-04-25T14:59:45.327660+0000","last_became_peered":"2024-04-25T14:59:45.327660+0000","last_unstale":"2024-04-25T15:00:02.717001+0000","last_undegraded":"2024-04-25T15:00:02.717001+0000","last_fullsized":"2024-04-25T15:00:02.717001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:29.862798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506904+0000","last_change":"2024-04-25T14:59:46.304038+0000","last_active":"2024-04-25T15:01:01.506904+0000","last_peered":"2024-04-25T15:01:01.506904+0000","last_clean":"2024-04-25T15:01:01.506904+0000","last_became_active":"2024-04-25T14:59:46.303713+0000","last_became_peered":"2024-04-25T14:59:46.303713+0000","last_unstale":"2024-04-25T15:01:01.506904+0000","last_undegraded":"2024-04-25T15:01:01.506904+0000","last_fullsized":"2024-04-25T15:01:01.506904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:30.476591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":220,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:10.821660+0000","last_change":"2024-04-25T14:59:45.315738+0000","last_active":"2024-04-25T15:01:10.821660+0000","last_peered":"2024-04-25T15:01:10.821660+0000","last_clean":"2024-04-25T15:01:10.821660+0000","last_became_active":"2024-04-25T14:59:45.315423+0000","last_became_peered":"2024-04-25T14:59:45.315423+0000","last_unstale":"2024-04-25T15:01:10.821660+0000","last_undegraded":"2024-04-25T15:01:10.821660+0000","last_fullsized":"2024-04-25T15:01:10.821660+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:33.135348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61295,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":172,"num_read_kb":145,"num_write":29,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.488835+0000","last_change":"2024-04-25T14:59:46.308085+0000","last_active":"2024-04-25T15:01:01.488835+0000","last_peered":"2024-04-25T15:01:01.488835+0000","last_clean":"2024-04-25T15:01:01.488835+0000","last_became_active":"2024-04-25T14:59:46.307739+0000","last_became_peered":"2024-04-25T14:59:46.307739+0000","last_unstale":"2024-04-25T15:01:01.488835+0000","last_undegraded":"2024-04-25T15:01:01.488835+0000","last_fullsized":"2024-04-25T15:01:01.488835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:57.949147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711939+0000","last_change":"2024-04-25T14:59:45.327058+0000","last_active":"2024-04-25T15:00:02.711939+0000","last_peered":"2024-04-25T15:00:02.711939+0000","last_clean":"2024-04-25T15:00:02.711939+0000","last_became_active":"2024-04-25T14:59:45.326817+0000","last_became_peered":"2024-04-25T14:59:45.326817+0000","last_unstale":"2024-04-25T15:00:02.711939+0000","last_undegraded":"2024-04-25T15:00:02.711939+0000","last_fullsized":"2024-04-25T15:00:02.711939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:51.087966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.512080+0000","last_change":"2024-04-25T14:59:46.328901+0000","last_active":"2024-04-25T15:01:01.512080+0000","last_peered":"2024-04-25T15:01:01.512080+0000","last_clean":"2024-04-25T15:01:01.512080+0000","last_became_active":"2024-04-25T14:59:46.328496+0000","last_became_peered":"2024-04-25T14:59:46.328496+0000","last_unstale":"2024-04-25T15:01:01.512080+0000","last_undegraded":"2024-04-25T15:01:01.512080+0000","last_fullsized":"2024-04-25T15:01:01.512080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:48.149719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716526+0000","last_change":"2024-04-25T14:59:45.316673+0000","last_active":"2024-04-25T15:00:02.716526+0000","last_peered":"2024-04-25T15:00:02.716526+0000","last_clean":"2024-04-25T15:00:02.716526+0000","last_became_active":"2024-04-25T14:59:45.316280+0000","last_became_peered":"2024-04-25T14:59:45.316280+0000","last_unstale":"2024-04-25T15:00:02.716526+0000","last_undegraded":"2024-04-25T15:00:02.716526+0000","last_fullsized":"2024-04-25T15:00:02.716526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:51.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508449+0000","last_change":"2024-04-25T14:59:46.323821+0000","last_active":"2024-04-25T15:01:01.508449+0000","last_peered":"2024-04-25T15:01:01.508449+0000","last_clean":"2024-04-25T15:01:01.508449+0000","last_became_active":"2024-04-25T14:59:46.323434+0000","last_became_peered":"2024-04-25T14:59:46.323434+0000","last_unstale":"2024-04-25T15:01:01.508449+0000","last_undegraded":"2024-04-25T15:01:01.508449+0000","last_fullsized":"2024-04-25T15:01:01.508449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:20.983270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.955733+0000","last_change":"2024-04-25T14:59:45.317953+0000","last_active":"2024-04-25T15:00:02.955733+0000","last_peered":"2024-04-25T15:00:02.955733+0000","last_clean":"2024-04-25T15:00:02.955733+0000","last_became_active":"2024-04-25T14:59:45.317617+0000","last_became_peered":"2024-04-25T14:59:45.317617+0000","last_unstale":"2024-04-25T15:00:02.955733+0000","last_undegraded":"2024-04-25T15:00:02.955733+0000","last_fullsized":"2024-04-25T15:00:02.955733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:44.235492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.500574+0000","last_change":"2024-04-25T14:59:46.305929+0000","last_active":"2024-04-25T15:01:01.500574+0000","last_peered":"2024-04-25T15:01:01.500574+0000","last_clean":"2024-04-25T15:01:01.500574+0000","last_became_active":"2024-04-25T14:59:46.305604+0000","last_became_peered":"2024-04-25T14:59:46.305604+0000","last_unstale":"2024-04-25T15:01:01.500574+0000","last_undegraded":"2024-04-25T15:01:01.500574+0000","last_fullsized":"2024-04-25T15:01:01.500574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:37.340009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.731349+0000","last_change":"2024-04-25T15:00:02.731349+0000","last_active":"2024-04-25T15:00:02.731349+0000","last_peered":"2024-04-25T15:00:02.731349+0000","last_clean":"2024-04-25T15:00:02.731349+0000","last_became_active":"2024-04-25T15:00:02.723113+0000","last_became_peered":"2024-04-25T15:00:02.723113+0000","last_unstale":"2024-04-25T15:00:02.731349+0000","last_undegraded":"2024-04-25T15:00:02.731349+0000","last_fullsized":"2024-04-25T15:00:02.731349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:35.964461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.496783+0000","last_change":"2024-04-25T15:00:03.077923+0000","last_active":"2024-04-25T15:01:01.496783+0000","last_peered":"2024-04-25T15:01:01.496783+0000","last_clean":"2024-04-25T15:01:01.496783+0000","last_became_active":"2024-04-25T15:00:03.076709+0000","last_became_peered":"2024-04-25T15:00:03.076709+0000","last_unstale":"2024-04-25T15:01:01.496783+0000","last_undegraded":"2024-04-25T15:01:01.496783+0000","last_fullsized":"2024-04-25T15:01:01.496783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:22.427477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711970+0000","last_change":"2024-04-25T14:59:45.318291+0000","last_active":"2024-04-25T15:00:02.711970+0000","last_peered":"2024-04-25T15:00:02.711970+0000","last_clean":"2024-04-25T15:00:02.711970+0000","last_became_active":"2024-04-25T14:59:45.317590+0000","last_became_peered":"2024-04-25T14:59:45.317590+0000","last_unstale":"2024-04-25T15:00:02.711970+0000","last_undegraded":"2024-04-25T15:00:02.711970+0000","last_fullsized":"2024-04-25T15:00:02.711970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.488749+0000","last_change":"2024-04-25T14:59:46.303503+0000","last_active":"2024-04-25T15:01:01.488749+0000","last_peered":"2024-04-25T15:01:01.488749+0000","last_clean":"2024-04-25T15:01:01.488749+0000","last_became_active":"2024-04-25T14:59:46.303127+0000","last_became_peered":"2024-04-25T14:59:46.303127+0000","last_unstale":"2024-04-25T15:01:01.488749+0000","last_undegraded":"2024-04-25T15:01:01.488749+0000","last_fullsized":"2024-04-25T15:01:01.488749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:46.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.721968+0000","last_change":"2024-04-25T14:59:45.330010+0000","last_active":"2024-04-25T15:00:02.721968+0000","last_peered":"2024-04-25T15:00:02.721968+0000","last_clean":"2024-04-25T15:00:02.721968+0000","last_became_active":"2024-04-25T14:59:45.329225+0000","last_became_peered":"2024-04-25T14:59:45.329225+0000","last_unstale":"2024-04-25T15:00:02.721968+0000","last_undegraded":"2024-04-25T15:00:02.721968+0000","last_fullsized":"2024-04-25T15:00:02.721968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:01.623458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.504077+0000","last_change":"2024-04-25T14:59:46.320003+0000","last_active":"2024-04-25T15:01:01.504077+0000","last_peered":"2024-04-25T15:01:01.504077+0000","last_clean":"2024-04-25T15:01:01.504077+0000","last_became_active":"2024-04-25T14:59:46.319600+0000","last_became_peered":"2024-04-25T14:59:46.319600+0000","last_unstale":"2024-04-25T15:01:01.504077+0000","last_undegraded":"2024-04-25T15:01:01.504077+0000","last_fullsized":"2024-04-25T15:01:01.504077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.916018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715203+0000","last_change":"2024-04-25T14:59:45.325233+0000","last_active":"2024-04-25T15:00:02.715203+0000","last_peered":"2024-04-25T15:00:02.715203+0000","last_clean":"2024-04-25T15:00:02.715203+0000","last_became_active":"2024-04-25T14:59:45.324826+0000","last_became_peered":"2024-04-25T14:59:45.324826+0000","last_unstale":"2024-04-25T15:00:02.715203+0000","last_undegraded":"2024-04-25T15:00:02.715203+0000","last_fullsized":"2024-04-25T15:00:02.715203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.422559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.513615+0000","last_change":"2024-04-25T14:59:46.314552+0000","last_active":"2024-04-25T15:01:01.513615+0000","last_peered":"2024-04-25T15:01:01.513615+0000","last_clean":"2024-04-25T15:01:01.513615+0000","last_became_active":"2024-04-25T14:59:46.314159+0000","last_became_peered":"2024-04-25T14:59:46.314159+0000","last_unstale":"2024-04-25T15:01:01.513615+0000","last_undegraded":"2024-04-25T15:01:01.513615+0000","last_fullsized":"2024-04-25T15:01:01.513615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:06.731960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.491375+0000","last_change":"2024-04-25T14:59:46.303354+0000","last_active":"2024-04-25T15:01:01.491375+0000","last_peered":"2024-04-25T15:01:01.491375+0000","last_clean":"2024-04-25T15:01:01.491375+0000","last_became_active":"2024-04-25T14:59:46.302869+0000","last_became_peered":"2024-04-25T14:59:46.302869+0000","last_unstale":"2024-04-25T15:01:01.491375+0000","last_undegraded":"2024-04-25T15:01:01.491375+0000","last_fullsized":"2024-04-25T15:01:01.491375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.796902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.984556+0000","last_change":"2024-04-25T14:59:45.336686+0000","last_active":"2024-04-25T15:00:02.984556+0000","last_peered":"2024-04-25T15:00:02.984556+0000","last_clean":"2024-04-25T15:00:02.984556+0000","last_became_active":"2024-04-25T14:59:45.335828+0000","last_became_peered":"2024-04-25T14:59:45.335828+0000","last_unstale":"2024-04-25T15:00:02.984556+0000","last_undegraded":"2024-04-25T15:00:02.984556+0000","last_fullsized":"2024-04-25T15:00:02.984556+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.676117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.509342+0000","last_change":"2024-04-25T14:59:46.319541+0000","last_active":"2024-04-25T15:01:01.509342+0000","last_peered":"2024-04-25T15:01:01.509342+0000","last_clean":"2024-04-25T15:01:01.509342+0000","last_became_active":"2024-04-25T14:59:46.317922+0000","last_became_peered":"2024-04-25T14:59:46.317922+0000","last_unstale":"2024-04-25T15:01:01.509342+0000","last_undegraded":"2024-04-25T15:01:01.509342+0000","last_fullsized":"2024-04-25T15:01:01.509342+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.454075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.955533+0000","last_change":"2024-04-25T14:59:45.313817+0000","last_active":"2024-04-25T15:00:02.955533+0000","last_peered":"2024-04-25T15:00:02.955533+0000","last_clean":"2024-04-25T15:00:02.955533+0000","last_became_active":"2024-04-25T14:59:45.313507+0000","last_became_peered":"2024-04-25T14:59:45.313507+0000","last_unstale":"2024-04-25T15:00:02.955533+0000","last_undegraded":"2024-04-25T15:00:02.955533+0000","last_fullsized":"2024-04-25T15:00:02.955533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:18.126351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715042+0000","last_change":"2024-04-25T14:58:29.935018+0000","last_active":"2024-04-25T15:00:02.715042+0000","last_peered":"2024-04-25T15:00:02.715042+0000","last_clean":"2024-04-25T15:00:02.715042+0000","last_became_active":"2024-04-25T14:58:29.934454+0000","last_became_peered":"2024-04-25T14:58:29.934454+0000","last_unstale":"2024-04-25T15:00:02.715042+0000","last_undegraded":"2024-04-25T15:00:02.715042+0000","last_fullsized":"2024-04-25T15:00:02.715042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_clean_scrub_stamp":"2024-04-25T14:54:46.455802+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:09.308783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.496987+0000","last_change":"2024-04-25T14:59:46.319680+0000","last_active":"2024-04-25T15:01:01.496987+0000","last_peered":"2024-04-25T15:01:01.496987+0000","last_clean":"2024-04-25T15:01:01.496987+0000","last_became_active":"2024-04-25T14:59:46.319225+0000","last_became_peered":"2024-04-25T14:59:46.319225+0000","last_unstale":"2024-04-25T15:01:01.496987+0000","last_undegraded":"2024-04-25T15:01:01.496987+0000","last_fullsized":"2024-04-25T15:01:01.496987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:34.629244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.715112+0000","last_change":"2024-04-25T14:59:45.318087+0000","last_active":"2024-04-25T15:00:02.715112+0000","last_peered":"2024-04-25T15:00:02.715112+0000","last_clean":"2024-04-25T15:00:02.715112+0000","last_became_active":"2024-04-25T14:59:45.317745+0000","last_became_peered":"2024-04-25T14:59:45.317745+0000","last_unstale":"2024-04-25T15:00:02.715112+0000","last_undegraded":"2024-04-25T15:00:02.715112+0000","last_fullsized":"2024-04-25T15:00:02.715112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:17.202201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711484+0000","last_change":"2024-04-25T14:59:45.332358+0000","last_active":"2024-04-25T15:00:02.711484+0000","last_peered":"2024-04-25T15:00:02.711484+0000","last_clean":"2024-04-25T15:00:02.711484+0000","last_became_active":"2024-04-25T14:59:45.332089+0000","last_became_peered":"2024-04-25T14:59:45.332089+0000","last_unstale":"2024-04-25T15:00:02.711484+0000","last_undegraded":"2024-04-25T15:00:02.711484+0000","last_fullsized":"2024-04-25T15:00:02.711484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:50.937548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.505595+0000","last_change":"2024-04-25T14:59:46.320945+0000","last_active":"2024-04-25T15:01:01.505595+0000","last_peered":"2024-04-25T15:01:01.505595+0000","last_clean":"2024-04-25T15:01:01.505595+0000","last_became_active":"2024-04-25T14:59:46.320602+0000","last_became_peered":"2024-04-25T14:59:46.320602+0000","last_unstale":"2024-04-25T15:01:01.505595+0000","last_undegraded":"2024-04-25T15:01:01.505595+0000","last_fullsized":"2024-04-25T15:01:01.505595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:32.282647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"88'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.724371+0000","last_change":"2024-04-25T14:59:45.334325+0000","last_active":"2024-04-25T15:00:02.724371+0000","last_peered":"2024-04-25T15:00:02.724371+0000","last_clean":"2024-04-25T15:00:02.724371+0000","last_became_active":"2024-04-25T14:59:45.333969+0000","last_became_peered":"2024-04-25T14:59:45.333969+0000","last_unstale":"2024-04-25T15:00:02.724371+0000","last_undegraded":"2024-04-25T15:00:02.724371+0000","last_fullsized":"2024-04-25T15:00:02.724371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:02.694267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.489178+0000","last_change":"2024-04-25T15:00:02.726558+0000","last_active":"2024-04-25T15:01:01.489178+0000","last_peered":"2024-04-25T15:01:01.489178+0000","last_clean":"2024-04-25T15:01:01.489178+0000","last_became_active":"2024-04-25T15:00:02.725467+0000","last_became_peered":"2024-04-25T15:00:02.725467+0000","last_unstale":"2024-04-25T15:01:01.489178+0000","last_undegraded":"2024-04-25T15:01:01.489178+0000","last_fullsized":"2024-04-25T15:01:01.489178+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.508211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.955688+0000","last_change":"2024-04-25T14:59:45.325553+0000","last_active":"2024-04-25T15:00:02.955688+0000","last_peered":"2024-04-25T15:00:02.955688+0000","last_clean":"2024-04-25T15:00:02.955688+0000","last_became_active":"2024-04-25T14:59:45.325201+0000","last_became_peered":"2024-04-25T14:59:45.325201+0000","last_unstale":"2024-04-25T15:00:02.955688+0000","last_undegraded":"2024-04-25T15:00:02.955688+0000","last_fullsized":"2024-04-25T15:00:02.955688+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:05.068269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.516873+0000","last_change":"2024-04-25T14:59:46.318252+0000","last_active":"2024-04-25T15:01:01.516873+0000","last_peered":"2024-04-25T15:01:01.516873+0000","last_clean":"2024-04-25T15:01:01.516873+0000","last_became_active":"2024-04-25T14:59:46.317601+0000","last_became_peered":"2024-04-25T14:59:46.317601+0000","last_unstale":"2024-04-25T15:01:01.516873+0000","last_undegraded":"2024-04-25T15:01:01.516873+0000","last_fullsized":"2024-04-25T15:01:01.516873+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:33.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:03.074978+0000","last_change":"2024-04-25T14:59:45.329430+0000","last_active":"2024-04-25T15:00:03.074978+0000","last_peered":"2024-04-25T15:00:03.074978+0000","last_clean":"2024-04-25T15:00:03.074978+0000","last_became_active":"2024-04-25T14:59:45.329140+0000","last_became_peered":"2024-04-25T14:59:45.329140+0000","last_unstale":"2024-04-25T15:00:03.074978+0000","last_undegraded":"2024-04-25T15:00:03.074978+0000","last_fullsized":"2024-04-25T15:00:03.074978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:49.114820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.459063+0000","last_change":"2024-04-25T14:59:46.299821+0000","last_active":"2024-04-25T15:01:01.459063+0000","last_peered":"2024-04-25T15:01:01.459063+0000","last_clean":"2024-04-25T15:01:01.459063+0000","last_became_active":"2024-04-25T14:59:46.299511+0000","last_became_peered":"2024-04-25T14:59:46.299511+0000","last_unstale":"2024-04-25T15:01:01.459063+0000","last_undegraded":"2024-04-25T15:01:01.459063+0000","last_fullsized":"2024-04-25T15:01:01.459063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:23.409333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711810+0000","last_change":"2024-04-25T14:59:45.331464+0000","last_active":"2024-04-25T15:00:02.711810+0000","last_peered":"2024-04-25T15:00:02.711810+0000","last_clean":"2024-04-25T15:00:02.711810+0000","last_became_active":"2024-04-25T14:59:45.327879+0000","last_became_peered":"2024-04-25T14:59:45.327879+0000","last_unstale":"2024-04-25T15:00:02.711810+0000","last_undegraded":"2024-04-25T15:00:02.711810+0000","last_fullsized":"2024-04-25T15:00:02.711810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:50.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.518028+0000","last_change":"2024-04-25T14:59:46.310621+0000","last_active":"2024-04-25T15:01:01.518028+0000","last_peered":"2024-04-25T15:01:01.518028+0000","last_clean":"2024-04-25T15:01:01.518028+0000","last_became_active":"2024-04-25T14:59:46.310292+0000","last_became_peered":"2024-04-25T14:59:46.310292+0000","last_unstale":"2024-04-25T15:01:01.518028+0000","last_undegraded":"2024-04-25T15:01:01.518028+0000","last_fullsized":"2024-04-25T15:01:01.518028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:20.035121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.713741+0000","last_change":"2024-04-25T15:00:02.713741+0000","last_active":"2024-04-25T15:00:02.713741+0000","last_peered":"2024-04-25T15:00:02.713741+0000","last_clean":"2024-04-25T15:00:02.713741+0000","last_became_active":"2024-04-25T15:00:02.713336+0000","last_became_peered":"2024-04-25T15:00:02.713336+0000","last_unstale":"2024-04-25T15:00:02.713741+0000","last_undegraded":"2024-04-25T15:00:02.713741+0000","last_fullsized":"2024-04-25T15:00:02.713741+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:37.967019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.515442+0000","last_change":"2024-04-25T14:59:46.326568+0000","last_active":"2024-04-25T15:01:01.515442+0000","last_peered":"2024-04-25T15:01:01.515442+0000","last_clean":"2024-04-25T15:01:01.515442+0000","last_became_active":"2024-04-25T14:59:46.324019+0000","last_became_peered":"2024-04-25T14:59:46.324019+0000","last_unstale":"2024-04-25T15:01:01.515442+0000","last_undegraded":"2024-04-25T15:01:01.515442+0000","last_fullsized":"2024-04-25T15:01:01.515442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.711345+0000","last_change":"2024-04-25T14:59:45.316935+0000","last_active":"2024-04-25T15:00:02.711345+0000","last_peered":"2024-04-25T15:00:02.711345+0000","last_clean":"2024-04-25T15:00:02.711345+0000","last_became_active":"2024-04-25T14:59:45.316604+0000","last_became_peered":"2024-04-25T14:59:45.316604+0000","last_unstale":"2024-04-25T15:00:02.711345+0000","last_undegraded":"2024-04-25T15:00:02.711345+0000","last_fullsized":"2024-04-25T15:00:02.711345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:45.699926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.506260+0000","last_change":"2024-04-25T14:59:46.316195+0000","last_active":"2024-04-25T15:01:01.506260+0000","last_peered":"2024-04-25T15:01:01.506260+0000","last_clean":"2024-04-25T15:01:01.506260+0000","last_became_active":"2024-04-25T14:59:46.315809+0000","last_became_peered":"2024-04-25T14:59:46.315809+0000","last_unstale":"2024-04-25T15:01:01.506260+0000","last_undegraded":"2024-04-25T15:01:01.506260+0000","last_fullsized":"2024-04-25T15:01:01.506260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:34.440058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.712878+0000","last_change":"2024-04-25T14:59:45.325137+0000","last_active":"2024-04-25T15:00:02.712878+0000","last_peered":"2024-04-25T15:00:02.712878+0000","last_clean":"2024-04-25T15:00:02.712878+0000","last_became_active":"2024-04-25T14:59:45.324769+0000","last_became_peered":"2024-04-25T14:59:45.324769+0000","last_unstale":"2024-04-25T15:00:02.712878+0000","last_undegraded":"2024-04-25T15:00:02.712878+0000","last_fullsized":"2024-04-25T15:00:02.712878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:02.778853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.498539+0000","last_change":"2024-04-25T14:59:46.304099+0000","last_active":"2024-04-25T15:01:01.498539+0000","last_peered":"2024-04-25T15:01:01.498539+0000","last_clean":"2024-04-25T15:01:01.498539+0000","last_became_active":"2024-04-25T14:59:46.303774+0000","last_became_peered":"2024-04-25T14:59:46.303774+0000","last_unstale":"2024-04-25T15:01:01.498539+0000","last_undegraded":"2024-04-25T15:01:01.498539+0000","last_fullsized":"2024-04-25T15:01:01.498539+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.308212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.716806+0000","last_change":"2024-04-25T14:59:45.322324+0000","last_active":"2024-04-25T15:00:02.716806+0000","last_peered":"2024-04-25T15:00:02.716806+0000","last_clean":"2024-04-25T15:00:02.716806+0000","last_became_active":"2024-04-25T14:59:45.322031+0000","last_became_peered":"2024-04-25T14:59:45.322031+0000","last_unstale":"2024-04-25T15:00:02.716806+0000","last_undegraded":"2024-04-25T15:00:02.716806+0000","last_fullsized":"2024-04-25T15:00:02.716806+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.424928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.495819+0000","last_change":"2024-04-25T15:00:02.730051+0000","last_active":"2024-04-25T15:01:01.495819+0000","last_peered":"2024-04-25T15:01:01.495819+0000","last_clean":"2024-04-25T15:01:01.495819+0000","last_became_active":"2024-04-25T15:00:02.725610+0000","last_became_peered":"2024-04-25T15:00:02.725610+0000","last_unstale":"2024-04-25T15:01:01.495819+0000","last_undegraded":"2024-04-25T15:01:01.495819+0000","last_fullsized":"2024-04-25T15:01:01.495819+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:04.598366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:00:02.985136+0000","last_change":"2024-04-25T14:59:45.335502+0000","last_active":"2024-04-25T15:00:02.985136+0000","last_peered":"2024-04-25T15:00:02.985136+0000","last_clean":"2024-04-25T15:00:02.985136+0000","last_became_active":"2024-04-25T14:59:45.335018+0000","last_became_peered":"2024-04-25T14:59:45.335018+0000","last_unstale":"2024-04-25T15:00:02.985136+0000","last_undegraded":"2024-04-25T15:00:02.985136+0000","last_fullsized":"2024-04-25T15:00:02.985136+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:11.918001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.501535+0000","last_change":"2024-04-25T14:59:46.304766+0000","last_active":"2024-04-25T15:01:01.501535+0000","last_peered":"2024-04-25T15:01:01.501535+0000","last_clean":"2024-04-25T15:01:01.501535+0000","last_became_active":"2024-04-25T14:59:46.304413+0000","last_became_peered":"2024-04-25T14:59:46.304413+0000","last_unstale":"2024-04-25T15:01:01.501535+0000","last_undegraded":"2024-04-25T15:01:01.501535+0000","last_fullsized":"2024-04-25T15:01:01.501535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.489458+0000","last_change":"2024-04-25T14:59:46.305285+0000","last_active":"2024-04-25T15:01:01.489458+0000","last_peered":"2024-04-25T15:01:01.489458+0000","last_clean":"2024-04-25T15:01:01.489458+0000","last_became_active":"2024-04-25T14:59:46.304934+0000","last_became_peered":"2024-04-25T14:59:46.304934+0000","last_unstale":"2024-04-25T15:01:01.489458+0000","last_undegraded":"2024-04-25T15:01:01.489458+0000","last_fullsized":"2024-04-25T15:01:01.489458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:44.892921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.497899+0000","last_change":"2024-04-25T15:00:02.718240+0000","last_active":"2024-04-25T15:01:01.497899+0000","last_peered":"2024-04-25T15:01:01.497899+0000","last_clean":"2024-04-25T15:01:01.497899+0000","last_became_active":"2024-04-25T15:00:02.717792+0000","last_became_peered":"2024-04-25T15:00:02.717792+0000","last_unstale":"2024-04-25T15:01:01.497899+0000","last_undegraded":"2024-04-25T15:01:01.497899+0000","last_fullsized":"2024-04-25T15:01:01.497899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:49.532532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.490203+0000","last_change":"2024-04-25T14:59:46.319694+0000","last_active":"2024-04-25T15:01:01.490203+0000","last_peered":"2024-04-25T15:01:01.490203+0000","last_clean":"2024-04-25T15:01:01.490203+0000","last_became_active":"2024-04-25T14:59:46.319417+0000","last_became_peered":"2024-04-25T14:59:46.319417+0000","last_unstale":"2024-04-25T15:01:01.490203+0000","last_undegraded":"2024-04-25T15:01:01.490203+0000","last_fullsized":"2024-04-25T15:01:01.490203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.177498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.513722+0000","last_change":"2024-04-25T15:00:02.986849+0000","last_active":"2024-04-25T15:01:01.513722+0000","last_peered":"2024-04-25T15:01:01.513722+0000","last_clean":"2024-04-25T15:01:01.513722+0000","last_became_active":"2024-04-25T15:00:02.986345+0000","last_became_peered":"2024-04-25T15:00:02.986345+0000","last_unstale":"2024-04-25T15:01:01.513722+0000","last_undegraded":"2024-04-25T15:01:01.513722+0000","last_fullsized":"2024-04-25T15:01:01.513722+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:22.707469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.517441+0000","last_change":"2024-04-25T14:59:46.319167+0000","last_active":"2024-04-25T15:01:01.517441+0000","last_peered":"2024-04-25T15:01:01.517441+0000","last_clean":"2024-04-25T15:01:01.517441+0000","last_became_active":"2024-04-25T14:59:46.318804+0000","last_became_peered":"2024-04-25T14:59:46.318804+0000","last_unstale":"2024-04-25T15:01:01.517441+0000","last_undegraded":"2024-04-25T15:01:01.517441+0000","last_fullsized":"2024-04-25T15:01:01.517441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:34.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.488823+0000","last_change":"2024-04-25T14:59:46.304554+0000","last_active":"2024-04-25T15:01:01.488823+0000","last_peered":"2024-04-25T15:01:01.488823+0000","last_clean":"2024-04-25T15:01:01.488823+0000","last_became_active":"2024-04-25T14:59:46.304062+0000","last_became_peered":"2024-04-25T14:59:46.304062+0000","last_unstale":"2024-04-25T15:01:01.488823+0000","last_undegraded":"2024-04-25T15:01:01.488823+0000","last_fullsized":"2024-04-25T15:01:01.488823+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.481095+0000","last_change":"2024-04-25T14:59:46.299104+0000","last_active":"2024-04-25T15:01:01.481095+0000","last_peered":"2024-04-25T15:01:01.481095+0000","last_clean":"2024-04-25T15:01:01.481095+0000","last_became_active":"2024-04-25T14:59:46.298765+0000","last_became_peered":"2024-04-25T14:59:46.298765+0000","last_unstale":"2024-04-25T15:01:01.481095+0000","last_undegraded":"2024-04-25T15:01:01.481095+0000","last_fullsized":"2024-04-25T15:01:01.481095+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.494298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.502770+0000","last_change":"2024-04-25T14:59:46.328885+0000","last_active":"2024-04-25T15:01:01.502770+0000","last_peered":"2024-04-25T15:01:01.502770+0000","last_clean":"2024-04-25T15:01:01.502770+0000","last_became_active":"2024-04-25T14:59:46.328375+0000","last_became_peered":"2024-04-25T14:59:46.328375+0000","last_unstale":"2024-04-25T15:01:01.502770+0000","last_undegraded":"2024-04-25T15:01:01.502770+0000","last_fullsized":"2024-04-25T15:01:01.502770+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:08.959045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.505955+0000","last_change":"2024-04-25T14:59:46.317403+0000","last_active":"2024-04-25T15:01:01.505955+0000","last_peered":"2024-04-25T15:01:01.505955+0000","last_clean":"2024-04-25T15:01:01.505955+0000","last_became_active":"2024-04-25T14:59:46.317100+0000","last_became_peered":"2024-04-25T14:59:46.317100+0000","last_unstale":"2024-04-25T15:01:01.505955+0000","last_undegraded":"2024-04-25T15:01:01.505955+0000","last_fullsized":"2024-04-25T15:01:01.505955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:23.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.509148+0000","last_change":"2024-04-25T14:59:46.327147+0000","last_active":"2024-04-25T15:01:01.509148+0000","last_peered":"2024-04-25T15:01:01.509148+0000","last_clean":"2024-04-25T15:01:01.509148+0000","last_became_active":"2024-04-25T14:59:46.321857+0000","last_became_peered":"2024-04-25T14:59:46.321857+0000","last_unstale":"2024-04-25T15:01:01.509148+0000","last_undegraded":"2024-04-25T15:01:01.509148+0000","last_fullsized":"2024-04-25T15:01:01.509148+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:46.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.508618+0000","last_change":"2024-04-25T14:59:46.310841+0000","last_active":"2024-04-25T15:01:01.508618+0000","last_peered":"2024-04-25T15:01:01.508618+0000","last_clean":"2024-04-25T15:01:01.508618+0000","last_became_active":"2024-04-25T14:59:46.310446+0000","last_became_peered":"2024-04-25T14:59:46.310446+0000","last_unstale":"2024-04-25T15:01:01.508618+0000","last_undegraded":"2024-04-25T15:01:01.508618+0000","last_fullsized":"2024-04-25T15:01:01.508618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.594784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.516706+0000","last_change":"2024-04-25T14:59:46.320715+0000","last_active":"2024-04-25T15:01:01.516706+0000","last_peered":"2024-04-25T15:01:01.516706+0000","last_clean":"2024-04-25T15:01:01.516706+0000","last_became_active":"2024-04-25T14:59:46.320238+0000","last_became_peered":"2024-04-25T14:59:46.320238+0000","last_unstale":"2024-04-25T15:01:01.516706+0000","last_undegraded":"2024-04-25T15:01:01.516706+0000","last_fullsized":"2024-04-25T15:01:01.516706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:55.387814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T15:01:01.484458+0000","last_change":"2024-04-25T14:59:46.303772+0000","last_active":"2024-04-25T15:01:01.484458+0000","last_peered":"2024-04-25T15:01:01.484458+0000","last_clean":"2024-04-25T15:01:01.484458+0000","last_became_active":"2024-04-25T14:59:46.303435+0000","last_became_peered":"2024-04-25T14:59:46.303435+0000","last_unstale":"2024-04-25T15:01:01.484458+0000","last_undegraded":"2024-04-25T15:01:01.484458+0000","last_fullsized":"2024-04-25T15:01:01.484458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:06.692979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62093,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":178,"num_read_kb":151,"num_write":80,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":186279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":458753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1110727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776169,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":962560,"data_stored":459452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":459357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28536,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716616,"statfs":{"total":95995035648,"available":95965814784,"internally_reserved":0,"allocated":1556480,"data_stored":1049820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":458870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":520048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":458843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":520082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":458974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":458775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1049420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":61329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:01:18.118 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T15:01:18.118 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T15:01:18.118 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T15:01:18.118 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T15:01:18.591 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:18.904 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:18 smithi094 ceph-mon[27862]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:01:18.905 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:18 smithi094 ceph-mon[27862]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-25T15:01:18.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:18 smithi138 ceph-mon[32875]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:01:18.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:18 smithi138 ceph-mon[32875]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-25T15:01:18.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:18 smithi137 ceph-mon[32439]: from='client.24898 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:01:18.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:18 smithi137 ceph-mon[32439]: pgmap v356: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-25T15:01:20.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 -- 172.21.15.94:0/4026471842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 msgr2=0x7f193c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 --2- 172.21.15.94:0/4026471842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 0x7f193c10f8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f193400b0a0 tx=0x7f193400b3b0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 -- 172.21.15.94:0/4026471842 shutdown_connections 2024-04-25T15:01:20.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 --2- 172.21.15.94:0/4026471842 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 0x7f193c10f8b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 --2- 172.21.15.94:0/4026471842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c108a50 0x7f193c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 --2- 172.21.15.94:0/4026471842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f193c102a90 0x7f193c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 -- 172.21.15.94:0/4026471842 >> 172.21.15.94:0/4026471842 conn(0x7f193c079ba0 msgr2=0x7f193c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:20.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 -- 172.21.15.94:0/4026471842 shutdown_connections 2024-04-25T15:01:20.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.727+0000 7f1942400700 1 -- 172.21.15.94:0/4026471842 wait complete. 2024-04-25T15:01:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.728+0000 7f1942400700 1 Processor -- start 2024-04-25T15:01:20.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.729+0000 7f1942400700 1 -- start start 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.729+0000 7f1942400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c102a90 0x7f193c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.729+0000 7f1942400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 0x7f193c19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.729+0000 7f1942400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f193c108a50 0x7f193c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.729+0000 7f1942400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f193c112c50 con 0x7f193c108a50 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.729+0000 7f1942400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f193c112ad0 con 0x7f193c103450 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.729+0000 7f1942400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f193c112dd0 con 0x7f193c102a90 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c102a90 0x7f193c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f1933e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 0x7f193c19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c102a90 0x7f193c19b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:44778/0 (socket says 172.21.15.94:44778) 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 -- 172.21.15.94:0/2697137505 learned_addr learned my addr 172.21.15.94:0/2697137505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193be00700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f193c108a50 0x7f193c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 -- 172.21.15.94:0/2697137505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 msgr2=0x7f193c19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 0x7f193c19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 -- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f193c108a50 msgr2=0x7f193c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f193c108a50 0x7f193c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 -- 172.21.15.94:0/2697137505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193c1a0330 con 0x7f193c102a90 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f1933e00700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 0x7f193c19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193be00700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f193c108a50 0x7f193c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.730+0000 7f193b400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c102a90 0x7f193c19b320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f192c00d950 tx=0x7f192c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.731+0000 7f1938c00700 1 -- 172.21.15.94:0/2697137505 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c016070 con 0x7f193c102a90 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.731+0000 7f1938c00700 1 -- 172.21.15.94:0/2697137505 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f192c00fd40 con 0x7f193c102a90 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.731+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193c1a0620 con 0x7f193c102a90 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.731+0000 7f1938c00700 1 -- 172.21.15.94:0/2697137505 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00e3b0 con 0x7f193c102a90 2024-04-25T15:01:20.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.731+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193c1a7ea0 con 0x7f193c102a90 2024-04-25T15:01:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.732+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f193c069730 con 0x7f193c102a90 2024-04-25T15:01:20.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.734+0000 7f1938c00700 1 -- 172.21.15.94:0/2697137505 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f192c019070 con 0x7f193c102a90 2024-04-25T15:01:20.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.735+0000 7f1938c00700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f19200a3cc0 0x7f19200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:20.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.735+0000 7f1938c00700 1 -- 172.21.15.94:0/2697137505 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f192c0cfdd0 con 0x7f193c102a90 2024-04-25T15:01:20.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.735+0000 7f1933e00700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f19200a3cc0 0x7f19200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:20.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.736+0000 7f1933e00700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f19200a3cc0 0x7f19200a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f193c07a540 tx=0x7f1924009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:20.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:20.740+0000 7f1938c00700 1 -- 172.21.15.94:0/2697137505 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f192c09c770 con 0x7f193c102a90 2024-04-25T15:01:20.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:20 smithi138 ceph-mon[32875]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-04-25T15:01:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:20 smithi094 ceph-mon[27862]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-04-25T15:01:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:20 smithi137 ceph-mon[32439]: pgmap v357: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-04-25T15:01:21.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.015+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f193c19c680 con 0x7f193c102a90 2024-04-25T15:01:21.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.016+0000 7f1938c00700 1 -- 172.21.15.94:0/2697137505 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f192c09c070 con 0x7f193c102a90 2024-04-25T15:01:21.017 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:21.017 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T15:01:21.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f19200a3cc0 msgr2=0x7f19200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:21.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f19200a3cc0 0x7f19200a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f193c07a540 tx=0x7f1924009260 comp rx=0 tx=0).stop 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c102a90 msgr2=0x7f193c19b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c102a90 0x7f193c19b320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f192c00d950 tx=0x7f192c00dc60 comp rx=0 tx=0).stop 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 shutdown_connections 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f19200a3cc0 0x7f19200a6180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f193c108a50 0x7f193c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193c102a90 0x7f193c19b320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 --2- 172.21.15.94:0/2697137505 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f193c103450 0x7f193c19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 >> 172.21.15.94:0/2697137505 conn(0x7f193c079ba0 msgr2=0x7f193c10dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 shutdown_connections 2024-04-25T15:01:21.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:21.019+0000 7f1942400700 1 -- 172.21.15.94:0/2697137505 wait complete. 2024-04-25T15:01:21.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:21 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2697137505' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:01:21.952 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T15:01:21.953 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:21.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:21 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2697137505' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:01:21.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:21 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2697137505' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:01:22.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 -- 172.21.15.94:0/1435835721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b81051c0 msgr2=0x7f49b8105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 --2- 172.21.15.94:0/1435835721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b81051c0 0x7f49b8105640 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f49a0009900 tx=0x7f49a002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 -- 172.21.15.94:0/1435835721 shutdown_connections 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 --2- 172.21.15.94:0/1435835721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b8105b80 0x7f49b810c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 --2- 172.21.15.94:0/1435835721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b81051c0 0x7f49b8105640 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 --2- 172.21.15.94:0/1435835721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49b8103fd0 0x7f49b81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 -- 172.21.15.94:0/1435835721 >> 172.21.15.94:0/1435835721 conn(0x7f49b80ff560 msgr2=0x7f49b8101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 -- 172.21.15.94:0/1435835721 shutdown_connections 2024-04-25T15:01:22.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.160+0000 7f49b7200640 1 -- 172.21.15.94:0/1435835721 wait complete. 2024-04-25T15:01:22.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.161+0000 7f49b7200640 1 Processor -- start 2024-04-25T15:01:22.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.161+0000 7f49b7200640 1 -- start start 2024-04-25T15:01:22.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.161+0000 7f49b7200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b8103fd0 0x7f49b819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b81051c0 0x7f49b819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b7200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49b8105b80 0x7f49b81a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b7200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f49b810fb60 con 0x7f49b8103fd0 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b7200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f49b810f9e0 con 0x7f49b8105b80 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49b810fce0 con 0x7f49b81051c0 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b8103fd0 0x7f49b819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b8103fd0 0x7f49b819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51230/0 (socket says 172.21.15.94:51230) 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b5e00640 1 -- 172.21.15.94:0/554645087 learned_addr learned my addr 172.21.15.94:0/554645087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b6800640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49b8105b80 0x7f49b81a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49ad400640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b81051c0 0x7f49b819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b6800640 1 -- 172.21.15.94:0/554645087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b81051c0 msgr2=0x7f49b819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b6800640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b81051c0 0x7f49b819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b6800640 1 -- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b8103fd0 msgr2=0x7f49b819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b6800640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b8103fd0 0x7f49b819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49b6800640 1 -- 172.21.15.94:0/554645087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b81a1e00 con 0x7f49b8105b80 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.162+0000 7f49ad400640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b81051c0 0x7f49b819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.163+0000 7f49b6800640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49b8105b80 0x7f49b81a1790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009fd0 tx=0x7f49b000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.163+0000 7f49af400640 1 -- 172.21.15.94:0/554645087 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b000ce20 con 0x7f49b8105b80 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.163+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b81a20f0 con 0x7f49b8105b80 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.163+0000 7f49af400640 1 -- 172.21.15.94:0/554645087 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b0005020 con 0x7f49b8105b80 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.163+0000 7f49af400640 1 -- 172.21.15.94:0/554645087 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0010690 con 0x7f49b8105b80 2024-04-25T15:01:22.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.163+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b81a2660 con 0x7f49b8105b80 2024-04-25T15:01:22.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.164+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b810b2f0 con 0x7f49b8105b80 2024-04-25T15:01:22.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.166+0000 7f49af400640 1 -- 172.21.15.94:0/554645087 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f49b00054d0 con 0x7f49b8105b80 2024-04-25T15:01:22.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.168+0000 7f49af400640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f499c0a3bb0 0x7f499c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.168+0000 7f49af400640 1 -- 172.21.15.94:0/554645087 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f49b00d0fa0 con 0x7f49b8105b80 2024-04-25T15:01:22.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.168+0000 7f49b5e00640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f499c0a3bb0 0x7f499c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.169+0000 7f49b5e00640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f499c0a3bb0 0x7f499c0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f49a80058c0 tx=0x7f49a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:22.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.170+0000 7f49af400640 1 -- 172.21.15.94:0/554645087 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49b009d940 con 0x7f49b8105b80 2024-04-25T15:01:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.382+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49b8062b50 con 0x7f49b8105b80 2024-04-25T15:01:22.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.384+0000 7f49af400640 1 -- 172.21.15.94:0/554645087 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7f49b009d240 con 0x7f49b8105b80 2024-04-25T15:01:22.386 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:22.386 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f499c0a3bb0 msgr2=0x7f499c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f499c0a3bb0 0x7f499c0a6070 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f49a80058c0 tx=0x7f49a8005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49b8105b80 msgr2=0x7f49b81a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49b8105b80 0x7f49b81a1790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009fd0 tx=0x7f49b000ef10 comp rx=0 tx=0).stop 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 shutdown_connections 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f499c0a3bb0 0x7f499c0a6070 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f49b8105b80 0x7f49b81a1790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49b81051c0 0x7f49b819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 --2- 172.21.15.94:0/554645087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f49b8103fd0 0x7f49b819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 >> 172.21.15.94:0/554645087 conn(0x7f49b80ff560 msgr2=0x7f49b81084c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 shutdown_connections 2024-04-25T15:01:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.387+0000 7f49b7200640 1 -- 172.21.15.94:0/554645087 wait complete. 2024-04-25T15:01:22.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:01:22.408 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:01:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 -- 172.21.15.94:0/2354100559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c1006d0 msgr2=0x7f8c0c102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 --2- 172.21.15.94:0/2354100559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c102b00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009860 tx=0x7f8c0002a340 comp rx=0 tx=0).stop 2024-04-25T15:01:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 -- 172.21.15.94:0/2354100559 shutdown_connections 2024-04-25T15:01:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 --2- 172.21.15.94:0/2354100559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c0c1059f0 0x7f8c0c105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 --2- 172.21.15.94:0/2354100559 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c103040 0x7f8c0c1054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 --2- 172.21.15.94:0/2354100559 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c102b00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 -- 172.21.15.94:0/2354100559 >> 172.21.15.94:0/2354100559 conn(0x7f8c0c0fc2f0 msgr2=0x7f8c0c0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.613+0000 7f8c12e00640 1 -- 172.21.15.94:0/2354100559 shutdown_connections 2024-04-25T15:01:22.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.614+0000 7f8c12e00640 1 -- 172.21.15.94:0/2354100559 wait complete. 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.614+0000 7f8c12e00640 1 Processor -- start 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12e00640 1 -- start start 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c103040 0x7f8c0c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c0c1059f0 0x7f8c0c1a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c113fb0 con 0x7f8c0c1006d0 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c113e30 con 0x7f8c0c103040 2024-04-25T15:01:22.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c114130 con 0x7f8c0c1059f0 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c11a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c11a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51252/0 (socket says 172.21.15.94:51252) 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c11a00640 1 -- 172.21.15.94:0/205915471 learned_addr learned my addr 172.21.15.94:0/205915471 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c11000640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c103040 0x7f8c0c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.615+0000 7f8c12400640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c0c1059f0 0x7f8c0c1a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c11a00640 1 -- 172.21.15.94:0/205915471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c0c1059f0 msgr2=0x7f8c0c1a5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c11a00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c0c1059f0 0x7f8c0c1a5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c11a00640 1 -- 172.21.15.94:0/205915471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c103040 msgr2=0x7f8c0c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c11a00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c103040 0x7f8c0c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c11a00640 1 -- 172.21.15.94:0/205915471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c1a6240 con 0x7f8c0c1006d0 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c11000640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c103040 0x7f8c0c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c11a00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c19e620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c000057c0 tx=0x7f8c00032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c0a000640 1 -- 172.21.15.94:0/205915471 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00037040 con 0x7f8c0c1006d0 2024-04-25T15:01:22.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c0a000640 1 -- 172.21.15.94:0/205915471 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c00030bd0 con 0x7f8c0c1006d0 2024-04-25T15:01:22.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c0a000640 1 -- 172.21.15.94:0/205915471 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00005280 con 0x7f8c0c1006d0 2024-04-25T15:01:22.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c1a64d0 con 0x7f8c0c1006d0 2024-04-25T15:01:22.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.616+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1a6910 con 0x7f8c0c1006d0 2024-04-25T15:01:22.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.618+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c0fdde0 con 0x7f8c0c1006d0 2024-04-25T15:01:22.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.620+0000 7f8c0a000640 1 -- 172.21.15.94:0/205915471 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8c00043020 con 0x7f8c0c1006d0 2024-04-25T15:01:22.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.621+0000 7f8c0a000640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8bf40a3bb0 0x7f8bf40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:22.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.621+0000 7f8c0a000640 1 -- 172.21.15.94:0/205915471 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f8c000efa90 con 0x7f8c0c1006d0 2024-04-25T15:01:22.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.621+0000 7f8c11000640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8bf40a3bb0 0x7f8bf40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:22.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.622+0000 7f8c11000640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8bf40a3bb0 0x7f8bf40a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c19fb30 tx=0x7f8bf8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:22.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.626+0000 7f8c0a000640 1 -- 172.21.15.94:0/205915471 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c0004a020 con 0x7f8c0c1006d0 2024-04-25T15:01:22.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:22 smithi094 ceph-mon[27862]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-04-25T15:01:22.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:22 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/554645087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.830+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8c0c062b50 con 0x7f8c0c1006d0 2024-04-25T15:01:22.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.831+0000 7f8c0a000640 1 -- 172.21.15.94:0/205915471 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f8c000bc430 con 0x7f8c0c1006d0 2024-04-25T15:01:22.833 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:01:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8bf40a3bb0 msgr2=0x7f8bf40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8bf40a3bb0 0x7f8bf40a6070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c19fb30 tx=0x7f8bf8005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c1006d0 msgr2=0x7f8c0c19e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c19e620 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c000057c0 tx=0x7f8c00032da0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 shutdown_connections 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f8bf40a3bb0 0x7f8bf40a6070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c0c1059f0 0x7f8c0c1a5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8c0c103040 0x7f8c0c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 --2- 172.21.15.94:0/205915471 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8c0c1006d0 0x7f8c0c19e620 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 >> 172.21.15.94:0/205915471 conn(0x7f8c0c0fc2f0 msgr2=0x7f8c0c103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 shutdown_connections 2024-04-25T15:01:22.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:22.834+0000 7f8c12e00640 1 -- 172.21.15.94:0/205915471 wait complete. 2024-04-25T15:01:22.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:01:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:22 smithi138 ceph-mon[32875]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-04-25T15:01:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:22 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/554645087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:22 smithi137 ceph-mon[32439]: pgmap v358: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 851 B/s rd, 1 op/s 2024-04-25T15:01:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:22 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/554645087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:23.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.051+0000 7f606ca00640 1 -- 172.21.15.94:0/396501503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6068075f90 msgr2=0x7f60680743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.051+0000 7f606ca00640 1 --2- 172.21.15.94:0/396501503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6068075f90 0x7f60680743e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6058009860 tx=0x7f605802a340 comp rx=0 tx=0).stop 2024-04-25T15:01:23.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.051+0000 7f606ca00640 1 -- 172.21.15.94:0/396501503 shutdown_connections 2024-04-25T15:01:23.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.051+0000 7f606ca00640 1 --2- 172.21.15.94:0/396501503 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6068109d20 0x7f606810c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.051+0000 7f606ca00640 1 --2- 172.21.15.94:0/396501503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f60680749b0 0x7f6068074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.051+0000 7f606ca00640 1 --2- 172.21.15.94:0/396501503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6068075f90 0x7f60680743e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.052+0000 7f606ca00640 1 -- 172.21.15.94:0/396501503 >> 172.21.15.94:0/396501503 conn(0x7f60680fd3f0 msgr2=0x7f60680ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:23.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.052+0000 7f606ca00640 1 -- 172.21.15.94:0/396501503 shutdown_connections 2024-04-25T15:01:23.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.052+0000 7f606ca00640 1 -- 172.21.15.94:0/396501503 wait complete. 2024-04-25T15:01:23.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.052+0000 7f606ca00640 1 Processor -- start 2024-04-25T15:01:23.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f606ca00640 1 -- start start 2024-04-25T15:01:23.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f606ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60680749b0 0x7f606819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f606ca00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6068075f90 0x7f606819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f606ca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6068109d20 0x7f60681a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f606ca00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f606810fb50 con 0x7f6068109d20 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f606ca00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f606810f9d0 con 0x7f6068075f90 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f606ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f606810fcd0 con 0x7f60680749b0 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f6067e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6068109d20 0x7f60681a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f605fe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6068075f90 0x7f606819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f6067e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6068109d20 0x7f60681a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51274/0 (socket says 172.21.15.94:51274) 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f6067e00640 1 -- 172.21.15.94:0/3129117410 learned_addr learned my addr 172.21.15.94:0/3129117410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.053+0000 7f6067400640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60680749b0 0x7f606819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6067e00640 1 -- 172.21.15.94:0/3129117410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60680749b0 msgr2=0x7f606819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6067e00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60680749b0 0x7f606819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6067e00640 1 -- 172.21.15.94:0/3129117410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6068075f90 msgr2=0x7f606819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6067e00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6068075f90 0x7f606819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6067e00640 1 -- 172.21.15.94:0/3129117410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60681a1e70 con 0x7f6068109d20 2024-04-25T15:01:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f605fe00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6068075f90 0x7f606819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6067e00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6068109d20 0x7f60681a1800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f606000eb10 tx=0x7f606000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6064c00640 1 -- 172.21.15.94:0/3129117410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606000ce20 con 0x7f6068109d20 2024-04-25T15:01:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60681a2160 con 0x7f6068109d20 2024-04-25T15:01:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6064c00640 1 -- 172.21.15.94:0/3129117410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6060005020 con 0x7f6068109d20 2024-04-25T15:01:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f6064c00640 1 -- 172.21.15.94:0/3129117410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6060010690 con 0x7f6068109d20 2024-04-25T15:01:23.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.054+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60681a2660 con 0x7f6068109d20 2024-04-25T15:01:23.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.055+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6068108a50 con 0x7f6068109d20 2024-04-25T15:01:23.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.058+0000 7f6064c00640 1 -- 172.21.15.94:0/3129117410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6060005510 con 0x7f6068109d20 2024-04-25T15:01:23.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.059+0000 7f6064c00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f604c0a3bb0 0x7f604c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.060+0000 7f6064c00640 1 -- 172.21.15.94:0/3129117410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f60600d09c0 con 0x7f6068109d20 2024-04-25T15:01:23.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.060+0000 7f6067400640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f604c0a3bb0 0x7f604c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.060+0000 7f6067400640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f604c0a3bb0 0x7f604c0a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6058000c00 tx=0x7f6058030470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.064+0000 7f6064c00640 1 -- 172.21.15.94:0/3129117410 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f606009d360 con 0x7f6068109d20 2024-04-25T15:01:23.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.265+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6068062b50 con 0x7f6068109d20 2024-04-25T15:01:23.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.266+0000 7f6064c00640 1 -- 172.21.15.94:0/3129117410 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f60600bbe20 con 0x7f6068109d20 2024-04-25T15:01:23.268 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:01:23.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.268+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f604c0a3bb0 msgr2=0x7f604c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.268+0000 7f606ca00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f604c0a3bb0 0x7f604c0a6070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6058000c00 tx=0x7f6058030470 comp rx=0 tx=0).stop 2024-04-25T15:01:23.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.268+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6068109d20 msgr2=0x7f60681a1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.268+0000 7f606ca00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6068109d20 0x7f60681a1800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f606000eb10 tx=0x7f606000ef10 comp rx=0 tx=0).stop 2024-04-25T15:01:23.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 shutdown_connections 2024-04-25T15:01:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f604c0a3bb0 0x7f604c0a6070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6068109d20 0x7f60681a1800 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f6068075f90 0x7f606819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 --2- 172.21.15.94:0/3129117410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60680749b0 0x7f606819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 >> 172.21.15.94:0/3129117410 conn(0x7f60680fd3f0 msgr2=0x7f6068105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 shutdown_connections 2024-04-25T15:01:23.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.269+0000 7f606ca00640 1 -- 172.21.15.94:0/3129117410 wait complete. 2024-04-25T15:01:23.289 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:01:23.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.484+0000 7fe497e00640 1 -- 172.21.15.94:0/2752237004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981006d0 msgr2=0x7fe498100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.484+0000 7fe497e00640 1 --2- 172.21.15.94:0/2752237004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981006d0 0x7fe498100af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe490009ca0 tx=0x7fe49002a8c0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.484+0000 7fe497e00640 1 -- 172.21.15.94:0/2752237004 shutdown_connections 2024-04-25T15:01:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.484+0000 7fe497e00640 1 --2- 172.21.15.94:0/2752237004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4981019d0 0x7fe498101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.484+0000 7fe497e00640 1 --2- 172.21.15.94:0/2752237004 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe498101030 0x7fe498101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.484+0000 7fe497e00640 1 --2- 172.21.15.94:0/2752237004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981006d0 0x7fe498100af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.484+0000 7fe497e00640 1 -- 172.21.15.94:0/2752237004 >> 172.21.15.94:0/2752237004 conn(0x7fe4980fc2f0 msgr2=0x7fe4980fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.485+0000 7fe497e00640 1 -- 172.21.15.94:0/2752237004 shutdown_connections 2024-04-25T15:01:23.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.485+0000 7fe497e00640 1 -- 172.21.15.94:0/2752237004 wait complete. 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 Processor -- start 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 -- start start 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4981006d0 0x7fe49819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe498101030 0x7fe49819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981019d0 0x7fe4981a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe49810fbb0 con 0x7fe4981019d0 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe49810fa30 con 0x7fe4981006d0 2024-04-25T15:01:23.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe49810fd30 con 0x7fe498101030 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe496a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4981006d0 0x7fe49819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe497400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981019d0 0x7fe4981a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe496a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4981006d0 0x7fe49819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40778/0 (socket says 172.21.15.94:40778) 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe496a00640 1 -- 172.21.15.94:0/4027123909 learned_addr learned my addr 172.21.15.94:0/4027123909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.486+0000 7fe496000640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe498101030 0x7fe49819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497400640 1 -- 172.21.15.94:0/4027123909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe498101030 msgr2=0x7fe49819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497400640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe498101030 0x7fe49819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497400640 1 -- 172.21.15.94:0/4027123909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4981006d0 msgr2=0x7fe49819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497400640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4981006d0 0x7fe49819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497400640 1 -- 172.21.15.94:0/4027123909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4981a1e90 con 0x7fe4981019d0 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497400640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981019d0 0x7fe4981a1820 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe48800e970 tx=0x7fe48800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe48f400640 1 -- 172.21.15.94:0/4027123909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48800ccc0 con 0x7fe4981019d0 2024-04-25T15:01:23.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe48f400640 1 -- 172.21.15.94:0/4027123909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe488005020 con 0x7fe4981019d0 2024-04-25T15:01:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe48f400640 1 -- 172.21.15.94:0/4027123909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488004a00 con 0x7fe4981019d0 2024-04-25T15:01:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4981a2180 con 0x7fe4981019d0 2024-04-25T15:01:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.487+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4981a2720 con 0x7fe4981019d0 2024-04-25T15:01:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.489+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49810b420 con 0x7fe4981019d0 2024-04-25T15:01:23.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.491+0000 7fe48f400640 1 -- 172.21.15.94:0/4027123909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe488004ba0 con 0x7fe4981019d0 2024-04-25T15:01:23.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.492+0000 7fe48f400640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe47c0a3c30 0x7fe47c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.492+0000 7fe48f400640 1 -- 172.21.15.94:0/4027123909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fe4880d0290 con 0x7fe4981019d0 2024-04-25T15:01:23.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.492+0000 7fe496a00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe47c0a3c30 0x7fe47c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.493+0000 7fe496a00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe47c0a3c30 0x7fe47c0a60f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe490009430 tx=0x7fe490009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.496+0000 7fe48f400640 1 -- 172.21.15.94:0/4027123909 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe48809cc70 con 0x7fe4981019d0 2024-04-25T15:01:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.687+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe498062b50 con 0x7fe4981019d0 2024-04-25T15:01:23.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.687+0000 7fe48f400640 1 -- 172.21.15.94:0/4027123909 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fe48809c570 con 0x7fe4981019d0 2024-04-25T15:01:23.689 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:01:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe47c0a3c30 msgr2=0x7fe47c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe47c0a3c30 0x7fe47c0a60f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe490009430 tx=0x7fe490009360 comp rx=0 tx=0).stop 2024-04-25T15:01:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981019d0 msgr2=0x7fe4981a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981019d0 0x7fe4981a1820 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe48800e970 tx=0x7fe48800ee30 comp rx=0 tx=0).stop 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 shutdown_connections 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fe47c0a3c30 0x7fe47c0a60f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4981019d0 0x7fe4981a1820 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe498101030 0x7fe49819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 --2- 172.21.15.94:0/4027123909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe4981006d0 0x7fe49819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 >> 172.21.15.94:0/4027123909 conn(0x7fe4980fc2f0 msgr2=0x7fe4980fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 shutdown_connections 2024-04-25T15:01:23.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.690+0000 7fe497e00640 1 -- 172.21.15.94:0/4027123909 wait complete. 2024-04-25T15:01:23.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.903+0000 7f7efc600640 1 -- 172.21.15.94:0/2825821859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41085b0 msgr2=0x7f7ef41089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.903+0000 7f7efc600640 1 --2- 172.21.15.94:0/2825821859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41085b0 0x7f7ef41089d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009ca0 tx=0x7f7ee802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.903+0000 7f7efc600640 1 -- 172.21.15.94:0/2825821859 shutdown_connections 2024-04-25T15:01:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.903+0000 7f7efc600640 1 --2- 172.21.15.94:0/2825821859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef410a160 0x7f7ef41109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.903+0000 7f7efc600640 1 --2- 172.21.15.94:0/2825821859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41097a0 0x7f7ef4109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.903+0000 7f7efc600640 1 --2- 172.21.15.94:0/2825821859 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41085b0 0x7f7ef41089d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.904+0000 7f7efc600640 1 -- 172.21.15.94:0/2825821859 >> 172.21.15.94:0/2825821859 conn(0x7f7ef4077570 msgr2=0x7f7ef4079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:23.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.904+0000 7f7efc600640 1 -- 172.21.15.94:0/2825821859 shutdown_connections 2024-04-25T15:01:23.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.904+0000 7f7efc600640 1 -- 172.21.15.94:0/2825821859 wait complete. 2024-04-25T15:01:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 Processor -- start 2024-04-25T15:01:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 -- start start 2024-04-25T15:01:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41085b0 0x7f7ef419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41097a0 0x7f7ef419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef410a160 0x7f7ef41a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4114120 con 0x7f7ef41085b0 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef4113fa0 con 0x7f7ef41097a0 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7efc600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef41142a0 con 0x7f7ef410a160 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.905+0000 7f7ef9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41085b0 0x7f7ef419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41085b0 0x7f7ef419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51306/0 (socket says 172.21.15.94:51306) 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 -- 172.21.15.94:0/4131338098 learned_addr learned my addr 172.21.15.94:0/4131338098 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7efa600640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef410a160 0x7f7ef41a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9200640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41097a0 0x7f7ef419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 -- 172.21.15.94:0/4131338098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef410a160 msgr2=0x7f7ef41a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef410a160 0x7f7ef41a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 -- 172.21.15.94:0/4131338098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41097a0 msgr2=0x7f7ef419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41097a0 0x7f7ef419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:23.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 -- 172.21.15.94:0/4131338098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef41a6440 con 0x7f7ef41085b0 2024-04-25T15:01:23.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef9c00640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41085b0 0x7f7ef419e820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8005e30 tx=0x7f7ee80317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.906+0000 7f7ef2000640 1 -- 172.21.15.94:0/4131338098 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8003bf0 con 0x7f7ef41085b0 2024-04-25T15:01:23.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.907+0000 7f7ef2000640 1 -- 172.21.15.94:0/4131338098 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee8003d90 con 0x7f7ef41085b0 2024-04-25T15:01:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.907+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef41a66d0 con 0x7f7ef41085b0 2024-04-25T15:01:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.907+0000 7f7ef2000640 1 -- 172.21.15.94:0/4131338098 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee802fc50 con 0x7f7ef41085b0 2024-04-25T15:01:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.907+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef41a6c20 con 0x7f7ef41085b0 2024-04-25T15:01:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.908+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef410f9a0 con 0x7f7ef41085b0 2024-04-25T15:01:23.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.910+0000 7f7ef2000640 1 -- 172.21.15.94:0/4131338098 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ee8004040 con 0x7f7ef41085b0 2024-04-25T15:01:23.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.911+0000 7f7ef2000640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7edc0a3bb0 0x7f7edc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:23.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.911+0000 7f7ef2000640 1 -- 172.21.15.94:0/4131338098 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7ee80f1750 con 0x7f7ef41085b0 2024-04-25T15:01:23.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.911+0000 7f7ef9200640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7edc0a3bb0 0x7f7edc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:23.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.912+0000 7f7ef9200640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7edc0a3bb0 0x7f7edc0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7ef419fd30 tx=0x7f7ee0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:23.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:23.915+0000 7f7ef2000640 1 -- 172.21.15.94:0/4131338098 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ee80be1c0 con 0x7f7ef41085b0 2024-04-25T15:01:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:23 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/205915471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:23.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:23 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3129117410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:23 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/205915471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:23.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:23 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3129117410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:23.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:23 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/205915471' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:23.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:23 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3129117410' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:24.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.158+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ef402d5c0 con 0x7f7ef41085b0 2024-04-25T15:01:24.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.160+0000 7f7ef2000640 1 -- 172.21.15.94:0/4131338098 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f7ee80bdac0 con 0x7f7ef41085b0 2024-04-25T15:01:24.161 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:24.162 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:01:24.162 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:01:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7edc0a3bb0 msgr2=0x7f7edc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7edc0a3bb0 0x7f7edc0a6070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7ef419fd30 tx=0x7f7ee0005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:24.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41085b0 msgr2=0x7f7ef419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41085b0 0x7f7ef419e820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8005e30 tx=0x7f7ee80317e0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 shutdown_connections 2024-04-25T15:01:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7edc0a3bb0 0x7f7edc0a6070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ef410a160 0x7f7ef41a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7ef41097a0 0x7f7ef419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 --2- 172.21.15.94:0/4131338098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ef41085b0 0x7f7ef419e820 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 >> 172.21.15.94:0/4131338098 conn(0x7f7ef4077570 msgr2=0x7f7ef410cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.162+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 shutdown_connections 2024-04-25T15:01:24.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.163+0000 7f7efc600640 1 -- 172.21.15.94:0/4131338098 wait complete. 2024-04-25T15:01:24.184 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 -- 172.21.15.94:0/3579097959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c076910 msgr2=0x7f5b2c076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 --2- 172.21.15.94:0/3579097959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c076910 0x7f5b2c076d30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009900 tx=0x7f5b2402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 -- 172.21.15.94:0/3579097959 shutdown_connections 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 --2- 172.21.15.94:0/3579097959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2c0784f0 0x7f5b2c078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 --2- 172.21.15.94:0/3579097959 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b2c077b50 0x7f5b2c077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 --2- 172.21.15.94:0/3579097959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c076910 0x7f5b2c076d30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 -- 172.21.15.94:0/3579097959 >> 172.21.15.94:0/3579097959 conn(0x7f5b2c0ff740 msgr2=0x7f5b2c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 -- 172.21.15.94:0/3579097959 shutdown_connections 2024-04-25T15:01:24.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.388+0000 7f5b32600640 1 -- 172.21.15.94:0/3579097959 wait complete. 2024-04-25T15:01:24.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.389+0000 7f5b32600640 1 Processor -- start 2024-04-25T15:01:24.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.389+0000 7f5b32600640 1 -- start start 2024-04-25T15:01:24.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.389+0000 7f5b32600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2c076910 0x7f5b2c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b32600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b2c077b50 0x7f5b2c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b32600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c0784f0 0x7f5b2c1a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b32600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c114130 con 0x7f5b2c0784f0 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b32600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c113fb0 con 0x7f5b2c077b50 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b32600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c1142b0 con 0x7f5b2c076910 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c0784f0 0x7f5b2c1a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2c076910 0x7f5b2c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c0784f0 0x7f5b2c1a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51322/0 (socket says 172.21.15.94:51322) 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 -- 172.21.15.94:0/2823066206 learned_addr learned my addr 172.21.15.94:0/2823066206 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2aa00640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b2c077b50 0x7f5b2c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 -- 172.21.15.94:0/2823066206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2c076910 msgr2=0x7f5b2c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2c076910 0x7f5b2c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 -- 172.21.15.94:0/2823066206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b2c077b50 msgr2=0x7f5b2c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b2c077b50 0x7f5b2c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 -- 172.21.15.94:0/2823066206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c1a6430 con 0x7f5b2c0784f0 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.390+0000 7f5b2be00640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c0784f0 0x7f5b2c1a5dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00e970 tx=0x7f5b1c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:24.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.391+0000 7f5b23e00640 1 -- 172.21.15.94:0/2823066206 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00ccc0 con 0x7f5b2c0784f0 2024-04-25T15:01:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.391+0000 7f5b23e00640 1 -- 172.21.15.94:0/2823066206 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b1c00ce60 con 0x7f5b2c0784f0 2024-04-25T15:01:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.391+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c1a6720 con 0x7f5b2c0784f0 2024-04-25T15:01:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.391+0000 7f5b23e00640 1 -- 172.21.15.94:0/2823066206 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c010430 con 0x7f5b2c0784f0 2024-04-25T15:01:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.391+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c1a6cc0 con 0x7f5b2c0784f0 2024-04-25T15:01:24.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.392+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c110cf0 con 0x7f5b2c0784f0 2024-04-25T15:01:24.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.394+0000 7f5b23e00640 1 -- 172.21.15.94:0/2823066206 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5b1c004f90 con 0x7f5b2c0784f0 2024-04-25T15:01:24.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.395+0000 7f5b23e00640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b100a3c30 0x7f5b100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.395+0000 7f5b23e00640 1 -- 172.21.15.94:0/2823066206 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5b1c0d0840 con 0x7f5b2c0784f0 2024-04-25T15:01:24.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.396+0000 7f5b2b400640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b100a3c30 0x7f5b100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.396+0000 7f5b2b400640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b100a3c30 0x7f5b100a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5b24031500 tx=0x7f5b24005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:24.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.399+0000 7f5b23e00640 1 -- 172.21.15.94:0/2823066206 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b1c09d1e0 con 0x7f5b2c0784f0 2024-04-25T15:01:24.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.586+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b2c062b50 con 0x7f5b2c0784f0 2024-04-25T15:01:24.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.589+0000 7f5b23e00640 1 -- 172.21.15.94:0/2823066206 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7f5b1c09cae0 con 0x7f5b2c0784f0 2024-04-25T15:01:24.591 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:24.591 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:24.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.591+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b100a3c30 msgr2=0x7f5b100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.591+0000 7f5b32600640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b100a3c30 0x7f5b100a60f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5b24031500 tx=0x7f5b24005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:24.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.591+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c0784f0 msgr2=0x7f5b2c1a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.591+0000 7f5b32600640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c0784f0 0x7f5b2c1a5dc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00e970 tx=0x7f5b1c00ed70 comp rx=0 tx=0).stop 2024-04-25T15:01:24.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 shutdown_connections 2024-04-25T15:01:24.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5b100a3c30 0x7f5b100a60f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b2c0784f0 0x7f5b2c1a5dc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5b2c077b50 0x7f5b2c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 --2- 172.21.15.94:0/2823066206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b2c076910 0x7f5b2c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 >> 172.21.15.94:0/2823066206 conn(0x7f5b2c0ff740 msgr2=0x7f5b2c100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:24.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 shutdown_connections 2024-04-25T15:01:24.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.592+0000 7f5b32600640 1 -- 172.21.15.94:0/2823066206 wait complete. 2024-04-25T15:01:24.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:01:24.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 -- 172.21.15.94:0/255003526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 msgr2=0x7fb2241023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 --2- 172.21.15.94:0/255003526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb2241023c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb214009860 tx=0x7fb21402a340 comp rx=0 tx=0).stop 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 -- 172.21.15.94:0/255003526 shutdown_connections 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 --2- 172.21.15.94:0/255003526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb224103b50 0x7fb22410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 --2- 172.21.15.94:0/255003526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb224103190 0x7fb224103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 --2- 172.21.15.94:0/255003526 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb2241023c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 -- 172.21.15.94:0/255003526 >> 172.21.15.94:0/255003526 conn(0x7fb2240fd530 msgr2=0x7fb2240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 -- 172.21.15.94:0/255003526 shutdown_connections 2024-04-25T15:01:24.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.809+0000 7fb228e00640 1 -- 172.21.15.94:0/255003526 wait complete. 2024-04-25T15:01:24.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 Processor -- start 2024-04-25T15:01:24.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 -- start start 2024-04-25T15:01:24.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb22419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb224103190 0x7fb22419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb224103b50 0x7fb2241a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410dae0 con 0x7fb224103b50 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410d960 con 0x7fb224101fa0 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.810+0000 7fb228e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb22410dc60 con 0x7fb224103190 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb22419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb222800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb224103b50 0x7fb2241a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb22419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40824/0 (socket says 172.21.15.94:40824) 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb224103190 0x7fb22419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 -- 172.21.15.94:0/1028817666 learned_addr learned my addr 172.21.15.94:0/1028817666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 -- 172.21.15.94:0/1028817666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb224103190 msgr2=0x7fb22419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb224103190 0x7fb22419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 -- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb224103b50 msgr2=0x7fb2241a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb224103b50 0x7fb2241a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221e00640 1 -- 172.21.15.94:0/1028817666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2241a2050 con 0x7fb224101fa0 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb221400640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb224103190 0x7fb22419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.811+0000 7fb222800640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb224103b50 0x7fb2241a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.812+0000 7fb221e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb22419a430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2140057c0 tx=0x7fb214032da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.812+0000 7fb21aa00640 1 -- 172.21.15.94:0/1028817666 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214037040 con 0x7fb224101fa0 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.812+0000 7fb21aa00640 1 -- 172.21.15.94:0/1028817666 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb214030bd0 con 0x7fb224101fa0 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.812+0000 7fb21aa00640 1 -- 172.21.15.94:0/1028817666 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214005280 con 0x7fb224101fa0 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.812+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2241a22e0 con 0x7fb224101fa0 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.812+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2241a2720 con 0x7fb224101fa0 2024-04-25T15:01:24.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.813+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2241a2980 con 0x7fb224101fa0 2024-04-25T15:01:24.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.815+0000 7fb21aa00640 1 -- 172.21.15.94:0/1028817666 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb214043020 con 0x7fb224101fa0 2024-04-25T15:01:24.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.816+0000 7fb21aa00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb2080a3bb0 0x7fb2080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:24.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.817+0000 7fb21aa00640 1 -- 172.21.15.94:0/1028817666 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb2140ef950 con 0x7fb224101fa0 2024-04-25T15:01:24.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.817+0000 7fb221400640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb2080a3bb0 0x7fb2080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:24.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.817+0000 7fb221400640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb2080a3bb0 0x7fb2080a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb22419b940 tx=0x7fb20c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:24.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:24.821+0000 7fb21aa00640 1 -- 172.21.15.94:0/1028817666 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb2140bc2f0 con 0x7fb224101fa0 2024-04-25T15:01:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:24 smithi138 ceph-mon[32875]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:24 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4027123909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:24 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4131338098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:24.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:24 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2823066206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:24 smithi094 ceph-mon[27862]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:24 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4027123909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:24 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4131338098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:24.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:24 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2823066206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:24 smithi137 ceph-mon[32439]: pgmap v359: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:24 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4027123909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:24 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4131338098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:24.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:24 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2823066206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:25.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.022+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb2241a2b50 con 0x7fb224101fa0 2024-04-25T15:01:25.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.023+0000 7fb21aa00640 1 -- 172.21.15.94:0/1028817666 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fb214030d80 con 0x7fb224101fa0 2024-04-25T15:01:25.024 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:01:25.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb2080a3bb0 msgr2=0x7fb2080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb2080a3bb0 0x7fb2080a6070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb22419b940 tx=0x7fb20c009260 comp rx=0 tx=0).stop 2024-04-25T15:01:25.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 msgr2=0x7fb22419a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb22419a430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2140057c0 tx=0x7fb214032da0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 shutdown_connections 2024-04-25T15:01:25.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb2080a3bb0 0x7fb2080a6070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb224103b50 0x7fb2241a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb224103190 0x7fb22419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 --2- 172.21.15.94:0/1028817666 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb224101fa0 0x7fb22419a430 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.025+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 >> 172.21.15.94:0/1028817666 conn(0x7fb2240fd530 msgr2=0x7fb224074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:25.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.026+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 shutdown_connections 2024-04-25T15:01:25.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.026+0000 7fb228e00640 1 -- 172.21.15.94:0/1028817666 wait complete. 2024-04-25T15:01:25.046 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 -- 172.21.15.94:0/3624269062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc105210 msgr2=0x7fbebc107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 --2- 172.21.15.94:0/3624269062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc105210 0x7fbebc107600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009900 tx=0x7fbea402a410 comp rx=0 tx=0).stop 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 -- 172.21.15.94:0/3624269062 shutdown_connections 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 --2- 172.21.15.94:0/3624269062 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc107b40 0x7fbebc109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 --2- 172.21.15.94:0/3624269062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc105210 0x7fbebc107600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 --2- 172.21.15.94:0/3624269062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbebc1028e0 0x7fbebc104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 -- 172.21.15.94:0/3624269062 >> 172.21.15.94:0/3624269062 conn(0x7fbebc0fc2f0 msgr2=0x7fbebc0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 -- 172.21.15.94:0/3624269062 shutdown_connections 2024-04-25T15:01:25.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.246+0000 7fbec2000640 1 -- 172.21.15.94:0/3624269062 wait complete. 2024-04-25T15:01:25.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.247+0000 7fbec2000640 1 Processor -- start 2024-04-25T15:01:25.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.247+0000 7fbec2000640 1 -- start start 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.247+0000 7fbec2000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc1028e0 0x7fbebc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbebc105210 0x7fbebc19a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec2000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc107b40 0x7fbebc1a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec2000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10d9a0 con 0x7fbebc107b40 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec2000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10d820 con 0x7fbebc1028e0 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbebc10db20 con 0x7fbebc105210 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc1028e0 0x7fbebc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbeb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbebc105210 0x7fbebc19a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc1028e0 0x7fbebc19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40834/0 (socket says 172.21.15.94:40834) 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 -- 172.21.15.94:0/5030089 learned_addr learned my addr 172.21.15.94:0/5030089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec1600640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc107b40 0x7fbebc1a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 -- 172.21.15.94:0/5030089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbebc105210 msgr2=0x7fbebc19a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbebc105210 0x7fbebc19a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 -- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc107b40 msgr2=0x7fbebc1a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc107b40 0x7fbebc1a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.248+0000 7fbec0c00640 1 -- 172.21.15.94:0/5030089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbebc1a1f20 con 0x7fbebc1028e0 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.249+0000 7fbec0c00640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc1028e0 0x7fbebc19a300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00eb90 tx=0x7fbeac00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.249+0000 7fbeba000640 1 -- 172.21.15.94:0/5030089 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac004d20 con 0x7fbebc1028e0 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.249+0000 7fbeba000640 1 -- 172.21.15.94:0/5030089 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbeac00ee90 con 0x7fbebc1028e0 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.249+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbebc1a2210 con 0x7fbebc1028e0 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.249+0000 7fbeba000640 1 -- 172.21.15.94:0/5030089 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac010670 con 0x7fbebc1028e0 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.249+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbebc1a26e0 con 0x7fbebc1028e0 2024-04-25T15:01:25.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.250+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbebc109690 con 0x7fbebc1028e0 2024-04-25T15:01:25.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.253+0000 7fbeba000640 1 -- 172.21.15.94:0/5030089 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbeac005020 con 0x7fbebc1028e0 2024-04-25T15:01:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.254+0000 7fbeba000640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbea00a3bb0 0x7fbea00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.254+0000 7fbeba000640 1 -- 172.21.15.94:0/5030089 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbeac0d0a60 con 0x7fbebc1028e0 2024-04-25T15:01:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.254+0000 7fbeb3e00640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbea00a3bb0 0x7fbea00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.255+0000 7fbeb3e00640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbea00a3bb0 0x7fbea00a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009230 tx=0x7fbea40314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.259+0000 7fbeba000640 1 -- 172.21.15.94:0/5030089 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbeac09d400 con 0x7fbebc1028e0 2024-04-25T15:01:25.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.458+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbebc062b50 con 0x7fbebc1028e0 2024-04-25T15:01:25.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.460+0000 7fbeba000640 1 -- 172.21.15.94:0/5030089 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fbeac09cd00 con 0x7fbebc1028e0 2024-04-25T15:01:25.461 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbea00a3bb0 msgr2=0x7fbea00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbea00a3bb0 0x7fbea00a6070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009230 tx=0x7fbea40314a0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc1028e0 msgr2=0x7fbebc19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc1028e0 0x7fbebc19a300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00eb90 tx=0x7fbeac00c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 shutdown_connections 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbea00a3bb0 0x7fbea00a6070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbebc107b40 0x7fbebc1a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbebc105210 0x7fbebc19a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 --2- 172.21.15.94:0/5030089 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbebc1028e0 0x7fbebc19a300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.462+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 >> 172.21.15.94:0/5030089 conn(0x7fbebc0fc2f0 msgr2=0x7fbebc102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:25.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.463+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 shutdown_connections 2024-04-25T15:01:25.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.463+0000 7fbec2000640 1 -- 172.21.15.94:0/5030089 wait complete. 2024-04-25T15:01:25.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 -- 172.21.15.94:0/3463047582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c1041b0 msgr2=0x7f2e8c1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 --2- 172.21.15.94:0/3463047582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c1041b0 0x7f2e8c1045d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009900 tx=0x7f2e7c02a410 comp rx=0 tx=0).stop 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 -- 172.21.15.94:0/3463047582 shutdown_connections 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 --2- 172.21.15.94:0/3463047582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c105d60 0x7f2e8c10c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 --2- 172.21.15.94:0/3463047582 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e8c1053a0 0x7f2e8c105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 --2- 172.21.15.94:0/3463047582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c1041b0 0x7f2e8c1045d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 -- 172.21.15.94:0/3463047582 >> 172.21.15.94:0/3463047582 conn(0x7f2e8c0ff740 msgr2=0x7f2e8c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 -- 172.21.15.94:0/3463047582 shutdown_connections 2024-04-25T15:01:25.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.693+0000 7f2e92800640 1 -- 172.21.15.94:0/3463047582 wait complete. 2024-04-25T15:01:25.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.694+0000 7f2e92800640 1 Processor -- start 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e92800640 1 -- start start 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e92800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c1041b0 0x7f2e8c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e92800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e8c1053a0 0x7f2e8c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e92800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c105d60 0x7f2e8c1a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e92800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10fd40 con 0x7f2e8c105d60 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e92800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10fbc0 con 0x7f2e8c1053a0 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e92800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8c10fec0 con 0x7f2e8c1041b0 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c1041b0 0x7f2e8c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e8be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c105d60 0x7f2e8c1a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e83e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e8c1053a0 0x7f2e8c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e8b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c1041b0 0x7f2e8c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:44934/0 (socket says 172.21.15.94:44934) 2024-04-25T15:01:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.695+0000 7f2e8b400640 1 -- 172.21.15.94:0/3044675543 learned_addr learned my addr 172.21.15.94:0/3044675543 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:25.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e8be00640 1 -- 172.21.15.94:0/3044675543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c1041b0 msgr2=0x7f2e8c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e8be00640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c1041b0 0x7f2e8c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e8be00640 1 -- 172.21.15.94:0/3044675543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e8c1053a0 msgr2=0x7f2e8c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e8be00640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e8c1053a0 0x7f2e8c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e8be00640 1 -- 172.21.15.94:0/3044675543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8c1a2000 con 0x7f2e8c105d60 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e8be00640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c105d60 0x7f2e8c1a1990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009fd0 tx=0x7f2e8400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e88c00640 1 -- 172.21.15.94:0/3044675543 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8400ce20 con 0x7f2e8c105d60 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e88c00640 1 -- 172.21.15.94:0/3044675543 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e84005020 con 0x7f2e8c105d60 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e88c00640 1 -- 172.21.15.94:0/3044675543 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84010690 con 0x7f2e8c105d60 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.696+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8c1a22f0 con 0x7f2e8c105d60 2024-04-25T15:01:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.697+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e8c1a27f0 con 0x7f2e8c105d60 2024-04-25T15:01:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.698+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8c10b5a0 con 0x7f2e8c105d60 2024-04-25T15:01:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.699+0000 7f2e88c00640 1 -- 172.21.15.94:0/3044675543 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2e840054d0 con 0x7f2e8c105d60 2024-04-25T15:01:25.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.701+0000 7f2e88c00640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2e700a3c30 0x7f2e700a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:25.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.702+0000 7f2e88c00640 1 -- 172.21.15.94:0/3044675543 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2e840d0a40 con 0x7f2e8c105d60 2024-04-25T15:01:25.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.702+0000 7f2e8b400640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2e700a3c30 0x7f2e700a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:25.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.703+0000 7f2e8b400640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2e700a3c30 0x7f2e700a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:25.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.706+0000 7f2e88c00640 1 -- 172.21.15.94:0/3044675543 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e840d2e20 con 0x7f2e8c105d60 2024-04-25T15:01:25.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.906+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2e8c062bf0 con 0x7f2e8c105d60 2024-04-25T15:01:25.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.907+0000 7f2e88c00640 1 -- 172.21.15.94:0/3044675543 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f2e8409d420 con 0x7f2e8c105d60 2024-04-25T15:01:25.908 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:01:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2e700a3c30 msgr2=0x7f2e700a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2e700a3c30 0x7f2e700a60f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c000c00 tx=0x7f2e7c005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c105d60 msgr2=0x7f2e8c1a1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c105d60 0x7f2e8c1a1990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009fd0 tx=0x7f2e8400ef10 comp rx=0 tx=0).stop 2024-04-25T15:01:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 shutdown_connections 2024-04-25T15:01:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f2e700a3c30 0x7f2e700a60f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2e8c105d60 0x7f2e8c1a1990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2e8c1053a0 0x7f2e8c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 --2- 172.21.15.94:0/3044675543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e8c1041b0 0x7f2e8c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.909+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 >> 172.21.15.94:0/3044675543 conn(0x7f2e8c0ff740 msgr2=0x7f2e8c108770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.910+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 shutdown_connections 2024-04-25T15:01:25.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:25.910+0000 7f2e92800640 1 -- 172.21.15.94:0/3044675543 wait complete. 2024-04-25T15:01:25.927 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:25 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1028817666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:25.927 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:25 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/5030089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:25.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:25.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:25 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1028817666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:25.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:25 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/5030089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:25.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:25 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1028817666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:25.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:25 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/5030089' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 -- 172.21.15.94:0/1825585457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941083d0 msgr2=0x7fc1941087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 --2- 172.21.15.94:0/1825585457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941083d0 0x7fc1941087f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc184009900 tx=0x7fc18402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 -- 172.21.15.94:0/1825585457 shutdown_connections 2024-04-25T15:01:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 --2- 172.21.15.94:0/1825585457 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc194109f80 0x7fc194110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 --2- 172.21.15.94:0/1825585457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1941095c0 0x7fc194109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 --2- 172.21.15.94:0/1825585457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941083d0 0x7fc1941087f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 -- 172.21.15.94:0/1825585457 >> 172.21.15.94:0/1825585457 conn(0x7fc194077390 msgr2=0x7fc1940797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 -- 172.21.15.94:0/1825585457 shutdown_connections 2024-04-25T15:01:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.131+0000 7fc199800640 1 -- 172.21.15.94:0/1825585457 wait complete. 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.132+0000 7fc199800640 1 Processor -- start 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc199800640 1 -- start start 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc199800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1941083d0 0x7fc19419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc199800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941095c0 0x7fc19419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc199800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc194109f80 0x7fc1941a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc199800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc194113f60 con 0x7fc194109f80 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc199800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc194113de0 con 0x7fc1941083d0 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc199800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1941140e0 con 0x7fc1941095c0 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc198e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc194109f80 0x7fc1941a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc193e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1941083d0 0x7fc19419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc193e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1941083d0 0x7fc19419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:40868/0 (socket says 172.21.15.94:40868) 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc198e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc194109f80 0x7fc1941a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51394/0 (socket says 172.21.15.94:51394) 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc193e00640 1 -- 172.21.15.94:0/1879045529 learned_addr learned my addr 172.21.15.94:0/1879045529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.133+0000 7fc18b400640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941095c0 0x7fc19419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc193e00640 1 -- 172.21.15.94:0/1879045529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941095c0 msgr2=0x7fc19419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc193e00640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941095c0 0x7fc19419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc193e00640 1 -- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc194109f80 msgr2=0x7fc1941a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc193e00640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc194109f80 0x7fc1941a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc193e00640 1 -- 172.21.15.94:0/1879045529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1941a6230 con 0x7fc1941083d0 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc193e00640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1941083d0 0x7fc19419e610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc184031500 tx=0x7fc184003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc191600640 1 -- 172.21.15.94:0/1879045529 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18403f070 con 0x7fc1941083d0 2024-04-25T15:01:26.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc191600640 1 -- 172.21.15.94:0/1879045529 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc184031c50 con 0x7fc1941083d0 2024-04-25T15:01:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1941a64c0 con 0x7fc1941083d0 2024-04-25T15:01:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc191600640 1 -- 172.21.15.94:0/1879045529 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184035030 con 0x7fc1941083d0 2024-04-25T15:01:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.134+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1941a69b0 con 0x7fc1941083d0 2024-04-25T15:01:26.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.135+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc19410f740 con 0x7fc1941083d0 2024-04-25T15:01:26.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.138+0000 7fc191600640 1 -- 172.21.15.94:0/1879045529 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc184042070 con 0x7fc1941083d0 2024-04-25T15:01:26.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.139+0000 7fc191600640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1780a3bb0 0x7fc1780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.139+0000 7fc191600640 1 -- 172.21.15.94:0/1879045529 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc1840f11b0 con 0x7fc1941083d0 2024-04-25T15:01:26.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.139+0000 7fc18b400640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1780a3bb0 0x7fc1780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.140+0000 7fc18b400640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1780a3bb0 0x7fc1780a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc19419fb20 tx=0x7fc17c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:26.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.141+0000 7fc191600640 1 -- 172.21.15.94:0/1879045529 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc1840bd540 con 0x7fc1941083d0 2024-04-25T15:01:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.385+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc19402d5c0 con 0x7fc1941083d0 2024-04-25T15:01:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.387+0000 7fc191600640 1 -- 172.21.15.94:0/1879045529 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7fc1840c1840 con 0x7fc1941083d0 2024-04-25T15:01:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:26.388 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:01:26.389 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:01:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1780a3bb0 msgr2=0x7fc1780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1780a3bb0 0x7fc1780a6070 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc19419fb20 tx=0x7fc17c005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1941083d0 msgr2=0x7fc19419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1941083d0 0x7fc19419e610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc184031500 tx=0x7fc184003c10 comp rx=0 tx=0).stop 2024-04-25T15:01:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 shutdown_connections 2024-04-25T15:01:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1780a3bb0 0x7fc1780a6070 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc194109f80 0x7fc1941a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1941095c0 0x7fc19419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 --2- 172.21.15.94:0/1879045529 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1941083d0 0x7fc19419e610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 >> 172.21.15.94:0/1879045529 conn(0x7fc194077390 msgr2=0x7fc19410c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 shutdown_connections 2024-04-25T15:01:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.390+0000 7fc199800640 1 -- 172.21.15.94:0/1879045529 wait complete. 2024-04-25T15:01:26.409 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 -- 172.21.15.94:0/668838954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd00ff9e0 msgr2=0x7f1dd00ffde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 --2- 172.21.15.94:0/668838954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd00ffde0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009860 tx=0x7f1dc002a340 comp rx=0 tx=0).stop 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 -- 172.21.15.94:0/668838954 shutdown_connections 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 --2- 172.21.15.94:0/668838954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd0107af0 0x7f1dd0109ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 --2- 172.21.15.94:0/668838954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd01003b0 0x7f1dd01075b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 --2- 172.21.15.94:0/668838954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd00ffde0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 -- 172.21.15.94:0/668838954 >> 172.21.15.94:0/668838954 conn(0x7f1dd00fb580 msgr2=0x7f1dd00fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 -- 172.21.15.94:0/668838954 shutdown_connections 2024-04-25T15:01:26.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.611+0000 7f1dd5200640 1 -- 172.21.15.94:0/668838954 wait complete. 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 Processor -- start 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 -- start start 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd01003b0 0x7f1dd019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0107af0 0x7f1dd01a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd010d900 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd010d780 con 0x7f1dd0107af0 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dd5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd010da80 con 0x7f1dd01003b0 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dcf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.613+0000 7f1dcf400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd019a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51412/0 (socket says 172.21.15.94:51412) 2024-04-25T15:01:26.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcf400640 1 -- 172.21.15.94:0/3292760544 learned_addr learned my addr 172.21.15.94:0/3292760544 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:26.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcfe00640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0107af0 0x7f1dd01a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dc6a00640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd01003b0 0x7f1dd019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcf400640 1 -- 172.21.15.94:0/3292760544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd01003b0 msgr2=0x7f1dd019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcf400640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd01003b0 0x7f1dd019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcf400640 1 -- 172.21.15.94:0/3292760544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0107af0 msgr2=0x7f1dd01a17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcf400640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0107af0 0x7f1dd01a17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcf400640 1 -- 172.21.15.94:0/3292760544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd01a1e30 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.614+0000 7f1dcf400640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd019a260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00057c0 tx=0x7f1dc0032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.615+0000 7f1dccc00640 1 -- 172.21.15.94:0/3292760544 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0037040 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.615+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd01a20c0 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.615+0000 7f1dccc00640 1 -- 172.21.15.94:0/3292760544 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc0030bd0 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.615+0000 7f1dccc00640 1 -- 172.21.15.94:0/3292760544 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0005280 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.615+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01a25a0 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.616+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd01a28c0 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.618+0000 7f1dccc00640 1 -- 172.21.15.94:0/3292760544 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1dc0043020 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.619+0000 7f1dccc00640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1db40a3bb0 0x7f1db40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:26.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.619+0000 7f1dccc00640 1 -- 172.21.15.94:0/3292760544 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f1dc00efaa0 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.619+0000 7f1dc6a00640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1db40a3bb0 0x7f1db40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:26.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.620+0000 7f1dc6a00640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1db40a3bb0 0x7f1db40a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1dd019b720 tx=0x7f1db8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:26.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.624+0000 7f1dccc00640 1 -- 172.21.15.94:0/3292760544 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1dc00bc440 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:26 smithi094 ceph-mon[27862]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:26.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:26 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3044675543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:26.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:26 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1879045529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:26.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.825+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1dd01a2a90 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.830+0000 7f1dccc00640 1 -- 172.21.15.94:0/3292760544 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7f1dc00bbd40 con 0x7f1dd00ff9e0 2024-04-25T15:01:26.832 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:26.832 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:26.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.832+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1db40a3bb0 msgr2=0x7f1db40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.832+0000 7f1dd5200640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1db40a3bb0 0x7f1db40a6070 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1dd019b720 tx=0x7f1db8009260 comp rx=0 tx=0).stop 2024-04-25T15:01:26.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.832+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd00ff9e0 msgr2=0x7f1dd019a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:26.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.832+0000 7f1dd5200640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd019a260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00057c0 tx=0x7f1dc0032da0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 shutdown_connections 2024-04-25T15:01:26.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f1db40a3bb0 0x7f1db40a6070 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1dd0107af0 0x7f1dd01a17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd01003b0 0x7f1dd019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 --2- 172.21.15.94:0/3292760544 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1dd00ff9e0 0x7f1dd019a260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:26.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 >> 172.21.15.94:0/3292760544 conn(0x7f1dd00fb580 msgr2=0x7f1dd00fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:26.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 shutdown_connections 2024-04-25T15:01:26.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:26.833+0000 7f1dd5200640 1 -- 172.21.15.94:0/3292760544 wait complete. 2024-04-25T15:01:26.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:26 smithi138 ceph-mon[32875]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:26 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3044675543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:26 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1879045529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:26 smithi137 ceph-mon[32439]: pgmap v360: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:26 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3044675543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:26 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1879045529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:27.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 -- 172.21.15.94:0/3302430911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00941097a0 msgr2=0x7f0094109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 --2- 172.21.15.94:0/3302430911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00941097a0 0x7f0094109c20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0080009900 tx=0x7f008002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 -- 172.21.15.94:0/3302430911 shutdown_connections 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 --2- 172.21.15.94:0/3302430911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f009410a160 0x7f00941109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 --2- 172.21.15.94:0/3302430911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00941097a0 0x7f0094109c20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 --2- 172.21.15.94:0/3302430911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00941085b0 0x7f00941089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 -- 172.21.15.94:0/3302430911 >> 172.21.15.94:0/3302430911 conn(0x7f0094077570 msgr2=0x7f0094079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 -- 172.21.15.94:0/3302430911 shutdown_connections 2024-04-25T15:01:27.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.051+0000 7f009c200640 1 -- 172.21.15.94:0/3302430911 wait complete. 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 Processor -- start 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 -- start start 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00941085b0 0x7f009419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00941097a0 0x7f009419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f009410a160 0x7f00941a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0094114110 con 0x7f009410a160 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0094113f90 con 0x7f00941085b0 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.052+0000 7f009c200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0094114290 con 0x7f00941097a0 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f009410a160 0x7f00941a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f009410a160 0x7f00941a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34086/0 (socket says 172.21.15.94:34086) 2024-04-25T15:01:27.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 -- 172.21.15.94:0/2872439060 learned_addr learned my addr 172.21.15.94:0/2872439060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f0099800640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00941085b0 0x7f009419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f0098e00640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00941097a0 0x7f009419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 -- 172.21.15.94:0/2872439060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00941097a0 msgr2=0x7f009419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00941097a0 0x7f009419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 -- 172.21.15.94:0/2872439060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00941085b0 msgr2=0x7f009419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00941085b0 0x7f009419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f009a200640 1 -- 172.21.15.94:0/2872439060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00941a6430 con 0x7f009410a160 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.053+0000 7f0099800640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00941085b0 0x7f009419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.054+0000 7f009a200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f009410a160 0x7f00941a5dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f008c00eb10 tx=0x7f008c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.054+0000 7f0092000640 1 -- 172.21.15.94:0/2872439060 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c004b20 con 0x7f009410a160 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.054+0000 7f0092000640 1 -- 172.21.15.94:0/2872439060 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f008c004cc0 con 0x7f009410a160 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.054+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00941a6720 con 0x7f009410a160 2024-04-25T15:01:27.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.054+0000 7f0092000640 1 -- 172.21.15.94:0/2872439060 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c010670 con 0x7f009410a160 2024-04-25T15:01:27.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.054+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00941a6c40 con 0x7f009410a160 2024-04-25T15:01:27.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.055+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009410f970 con 0x7f009410a160 2024-04-25T15:01:27.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.057+0000 7f0092000640 1 -- 172.21.15.94:0/2872439060 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f008c005020 con 0x7f009410a160 2024-04-25T15:01:27.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.058+0000 7f0092000640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f007c0a3c30 0x7f007c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.058+0000 7f0092000640 1 -- 172.21.15.94:0/2872439060 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f008c0d0930 con 0x7f009410a160 2024-04-25T15:01:27.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.059+0000 7f0099800640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f007c0a3c30 0x7f007c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.059+0000 7f0099800640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f007c0a3c30 0x7f007c0a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0094109600 tx=0x7f0088005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.063+0000 7f0092000640 1 -- 172.21.15.94:0/2872439060 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f008c09d310 con 0x7f009410a160 2024-04-25T15:01:27.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.298+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f009402d5c0 con 0x7f009410a160 2024-04-25T15:01:27.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.300+0000 7f0092000640 1 -- 172.21.15.94:0/2872439060 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f008c09cc10 con 0x7f009410a160 2024-04-25T15:01:27.301 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:27.301 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:01:27.302 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:01:27.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f007c0a3c30 msgr2=0x7f007c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f007c0a3c30 0x7f007c0a60f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0094109600 tx=0x7f0088005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:27.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f009410a160 msgr2=0x7f00941a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f009410a160 0x7f00941a5dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f008c00eb10 tx=0x7f008c00c2d0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 shutdown_connections 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f007c0a3c30 0x7f007c0a60f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f009410a160 0x7f00941a5dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00941097a0 0x7f009419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 --2- 172.21.15.94:0/2872439060 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f00941085b0 0x7f009419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 >> 172.21.15.94:0/2872439060 conn(0x7f0094077570 msgr2=0x7f009410cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 shutdown_connections 2024-04-25T15:01:27.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.302+0000 7f009c200640 1 -- 172.21.15.94:0/2872439060 wait complete. 2024-04-25T15:01:27.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T14:59:45.806095+0000', 'modified': '2024-04-25T14:59:50.088429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24596}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/3586316280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 3586316280}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 3586316280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/3444396688', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 3444396688}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 3444396688}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24596]} 2024-04-25T15:01:27.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T15:01:27.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T15:01:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.517+0000 7fc1b5200640 1 -- 172.21.15.94:0/334312078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01006d0 msgr2=0x7fc1b0100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.517+0000 7fc1b5200640 1 --2- 172.21.15.94:0/334312078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01006d0 0x7fc1b0100af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009900 tx=0x7fc1a002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.518+0000 7fc1b5200640 1 -- 172.21.15.94:0/334312078 shutdown_connections 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.518+0000 7fc1b5200640 1 --2- 172.21.15.94:0/334312078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b01019d0 0x7fc1b0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.518+0000 7fc1b5200640 1 --2- 172.21.15.94:0/334312078 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b0101030 0x7fc1b0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.518+0000 7fc1b5200640 1 --2- 172.21.15.94:0/334312078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01006d0 0x7fc1b0100af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.518+0000 7fc1b5200640 1 -- 172.21.15.94:0/334312078 >> 172.21.15.94:0/334312078 conn(0x7fc1b00fc2f0 msgr2=0x7fc1b00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.518+0000 7fc1b5200640 1 -- 172.21.15.94:0/334312078 shutdown_connections 2024-04-25T15:01:27.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.518+0000 7fc1b5200640 1 -- 172.21.15.94:0/334312078 wait complete. 2024-04-25T15:01:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 Processor -- start 2024-04-25T15:01:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 -- start start 2024-04-25T15:01:27.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b01006d0 0x7fc1b019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0101030 0x7fc1b019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01019d0 0x7fc1b01a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b010fc00 con 0x7fc1b01019d0 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b010fa80 con 0x7fc1b01006d0 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1b5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b010fd80 con 0x7fc1b0101030 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01019d0 0x7fc1b01a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01019d0 0x7fc1b01a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34106/0 (socket says 172.21.15.94:34106) 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1afe00640 1 -- 172.21.15.94:0/1936835390 learned_addr learned my addr 172.21.15.94:0/1936835390 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1af400640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b01006d0 0x7fc1b019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.519+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0101030 0x7fc1b019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1afe00640 1 -- 172.21.15.94:0/1936835390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0101030 msgr2=0x7fc1b019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1afe00640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0101030 0x7fc1b019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1afe00640 1 -- 172.21.15.94:0/1936835390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b01006d0 msgr2=0x7fc1b019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1afe00640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b01006d0 0x7fc1b019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1afe00640 1 -- 172.21.15.94:0/1936835390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b01a1e60 con 0x7fc1b01019d0 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1af400640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b01006d0 0x7fc1b019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1afe00640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01019d0 0x7fc1b01a17f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800eb10 tx=0x7fc1a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1acc00640 1 -- 172.21.15.94:0/1936835390 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8004b20 con 0x7fc1b01019d0 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1acc00640 1 -- 172.21.15.94:0/1936835390 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a8004cc0 con 0x7fc1b01019d0 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b01a2150 con 0x7fc1b01019d0 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.520+0000 7fc1acc00640 1 -- 172.21.15.94:0/1936835390 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8010670 con 0x7fc1b01019d0 2024-04-25T15:01:27.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.521+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b01a26f0 con 0x7fc1b01019d0 2024-04-25T15:01:27.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 Processor -- start 2024-04-25T15:01:27.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1acc00640 1 -- 172.21.15.94:0/1936835390 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc1a8005020 con 0x7fc1b01019d0 2024-04-25T15:01:27.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 -- start start 2024-04-25T15:01:27.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0150 0x562370af0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370af5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af5cb0 0x562370af8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562370a54b10 con 0x562370af0150 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x5623709fb2b0 con 0x562370af0b40 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.524+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562370a50ad0 con 0x562370af5cb0 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0150 0x562370af0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1af400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370af5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0150 0x562370af0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34114/0 (socket says 172.21.15.94:34114) 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1a6a00640 1 -- 172.21.15.94:0/1584460860 learned_addr learned my addr 172.21.15.94:0/1584460860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af5cb0 0x562370af8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1af400640 1 -- 172.21.15.94:0/1584460860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af5cb0 msgr2=0x562370af8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1af400640 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af5cb0 0x562370af8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1af400640 1 -- 172.21.15.94:0/1584460860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0150 msgr2=0x562370af0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1af400640 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0150 0x562370af0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1af400640 1 -- 172.21.15.94:0/1584460860 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623707672d0 con 0x562370af0b40 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af5cb0 0x562370af8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.525+0000 7fc1af400640 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370af5640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a9e0 tx=0x7fc1a00043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e7ed07c155a574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1acc00640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1940a3c30 0x7fc1940a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1a4c00640 1 -- 172.21.15.94:0/1584460860 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0035030 con 0x562370af0b40 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1a4c00640 1 -- 172.21.15.94:0/1584460860 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0040070 con 0x562370af0b40 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1a4c00640 1 -- 172.21.15.94:0/1584460860 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0031d00 con 0x562370af0b40 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1acc00640 1 -- 172.21.15.94:0/1936835390 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a80d08e0 con 0x7fc1b01019d0 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1940a3c30 0x7fc1940a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1584460860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 msgr2=0x562370af5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370af5640 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a9e0 tx=0x7fc1a00043f0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.526+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1940a3c30 0x7fc1940a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562370af13a0 tx=0x7fc198009850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1584460860 shutdown_connections 2024-04-25T15:01:27.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af5cb0 0x562370af8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370af5640 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1584460860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0150 0x562370af0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1584460860 >> 172.21.15.94:0/1584460860 conn(0x562370aeb7c0 msgr2=0x562370aebbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1584460860 shutdown_connections 2024-04-25T15:01:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1584460860 wait complete. 2024-04-25T15:01:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 -- start start 2024-04-25T15:01:27.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.527+0000 7fc1b95c6740 1 Processor -- start 2024-04-25T15:01:27.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370afa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b01850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562370af8c00 con 0x562370af0b40 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562370a0e570 con 0x562370af5cb0 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562370af8d80 con 0x562370af0150 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1b95c6740 10 client.0 _ll_register_callbacks cb 0x5623709dcfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b01850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370afa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1af400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b01850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:45134/0 (socket says 172.21.15.94:45134) 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 -- 172.21.15.94:0/674440632 learned_addr learned my addr 172.21.15.94:0/674440632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:27.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 -- 172.21.15.94:0/674440632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 msgr2=0x562370afa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 -- 172.21.15.94:0/674440632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 msgr2=0x562370afa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370afa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.528+0000 7fc1afe00640 1 -- 172.21.15.94:0/674440632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623708316b0 con 0x562370af5cb0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc1afe00640 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b01850 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8004f90 tx=0x7fc1a809c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997b4bed7580e421 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc19d600640 10 client.0 ms_handle_connect on v2:172.21.15.137:3300/0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc19d600640 1 -- 172.21.15.94:0/674440632 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a80a0f30 con 0x562370af5cb0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a0040070 con 0x562370af5cb0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc19d600640 1 -- 172.21.15.94:0/674440632 <== mon.2 v2:172.21.15.137:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a809cde0 con 0x562370af5cb0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc19d600640 1 -- 172.21.15.94:0/674440632 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc1a80a57f0 con 0x562370af5cb0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc1b95c6740 10 client.24992 fetch_fsmap learned FSMap version 19 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562370b04bf0 con 0x562370af5cb0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.529+0000 7fc19d600640 1 -- 172.21.15.94:0/674440632 <== mon.2 v2:172.21.15.137:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a0040070 con 0x562370af5cb0 2024-04-25T15:01:27.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc19d600640 1 -- 172.21.15.94:0/674440632 <== mon.2 v2:172.21.15.137:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fc1a80988d0 con 0x562370af5cb0 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc1b95c6740 10 client.24992 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc1b95c6740 10 client.24992 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc1b95c6740 20 client.24992 populate_metadata read hostname 'smithi094' 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b071e0 0x562370b08650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc1b95c6740 4 client.24992 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 --> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562370a62530 con 0x562370b071e0 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.530+0000 7fc1af400640 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b071e0 0x562370b08650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.531+0000 7fc1af400640 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b071e0 0x562370b08650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652d342ee0c690f7 server_cookie=d34436a879b36f67 in_seq=0 out_seq=0 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.531+0000 7fc19d600640 10 client.24992 ms_handle_connect on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc19d600640 1 -- 172.21.15.94:0/674440632 <== mds.0 v2:172.21.15.137:6838/3444396688 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562370a62530 con 0x562370b071e0 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc19d600640 10 client.24992 handle_command_reply: tid=0 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 client.24992 shutdown 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 msgr2=0x562370b01850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b01850 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8004f90 tx=0x7fc1a809c4e0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 shutdown_connections 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b01850 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370afa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b071e0 0x562370b08650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/674440632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc19d600640 0 client.24992 ms_handle_reset on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 >> 172.21.15.94:0/674440632 conn(0x562370aeb7c0 msgr2=0x562370aeaa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 shutdown_connections 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 1 -- 172.21.15.94:0/674440632 wait complete. 2024-04-25T15:01:27.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.532+0000 7fc1b95c6740 20 client.24992 trim_cache size 0 max 16384 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.552+0000 7fc1b95c6740 1 Processor -- start 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.552+0000 7fc1b95c6740 1 -- start start 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370b01810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af5cb0 0x562370afa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562370b06990 con 0x562370af5cb0 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562370b07060 con 0x562370af0b40 2024-04-25T15:01:27.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562370b07730 con 0x562370af0150 2024-04-25T15:01:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370b01810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370b01810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:45142/0 (socket says 172.21.15.94:45142) 2024-04-25T15:01:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1afe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af5cb0 0x562370afa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 -- 172.21.15.94:0/2718188655 learned_addr learned my addr 172.21.15.94:0/2718188655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1af400640 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 -- 172.21.15.94:0/2718188655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 msgr2=0x562370afa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 -- 172.21.15.94:0/2718188655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af5cb0 msgr2=0x562370afa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af5cb0 0x562370afa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.553+0000 7fc1a6a00640 1 -- 172.21.15.94:0/2718188655 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562370af1810 con 0x562370af0b40 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.554+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370b01810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc19800f660 tx=0x7fc198018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9369c2d9bb4b8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.554+0000 7fc19fe00640 1 -- 172.21.15.94:0/2718188655 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800fdd0 con 0x562370af0b40 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.554+0000 7fc19fe00640 1 -- 172.21.15.94:0/2718188655 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1980054d0 con 0x562370af0b40 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.554+0000 7fc19fe00640 1 -- 172.21.15.94:0/2718188655 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19802a460 con 0x562370af0b40 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 -- 172.21.15.94:0/2718188655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 msgr2=0x562370b01810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370b01810 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc19800f660 tx=0x7fc198018000 comp rx=0 tx=0).stop 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 -- 172.21.15.94:0/2718188655 shutdown_connections 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af5cb0 0x562370afa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af0b40 0x562370b01810 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/2718188655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370afa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 -- 172.21.15.94:0/2718188655 >> 172.21.15.94:0/2718188655 conn(0x562370aeb7c0 msgr2=0x562370af5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 -- 172.21.15.94:0/2718188655 shutdown_connections 2024-04-25T15:01:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 -- 172.21.15.94:0/2718188655 wait complete. 2024-04-25T15:01:27.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 -- start start 2024-04-25T15:01:27.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.555+0000 7fc1b95c6740 1 Processor -- start 2024-04-25T15:01:27.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370aff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370b69170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1b95c6740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b6b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x562370b084d0 con 0x562370af0b40 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x562370b07e00 con 0x562370af5cb0 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1b95c6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562370b08ba0 con 0x562370af0150 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1b95c6740 10 client.0 _ll_register_callbacks cb 0x7fc1a809c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370b69170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1af400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370aff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1a6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370b69170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34148/0 (socket says 172.21.15.94:34148) 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1a6a00640 1 -- 172.21.15.94:0/1963931960 learned_addr learned my addr 172.21.15.94:0/1963931960 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.556+0000 7fc1afe00640 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b6b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1af400640 1 -- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 msgr2=0x562370b6b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1af400640 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b6b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1af400640 1 -- 172.21.15.94:0/1963931960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 msgr2=0x562370b69170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1af400640 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370b69170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1af400640 1 -- 172.21.15.94:0/1963931960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562370b6be50 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370b69170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1af400640 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370aff760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a9e0 tx=0x7fc1a00058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5643fd833a62f199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc19d600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc19d600640 1 -- 172.21.15.94:0/1963931960 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0035040 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1980054d0 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc19d600640 1 -- 172.21.15.94:0/1963931960 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a0005020 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc19d600640 1 -- 172.21.15.94:0/1963931960 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc1a0005220 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1b95c6740 10 client.24991 fetch_fsmap learned FSMap version 19 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.557+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562370b6d150 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc19d600640 1 -- 172.21.15.94:0/1963931960 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1980054d0 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc19d600640 1 -- 172.21.15.94:0/1963931960 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fc1a00035f0 con 0x562370af0150 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc1b95c6740 10 client.24991 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T15:01:27.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc1b95c6740 10 client.24991 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc1b95c6740 20 client.24991 populate_metadata read hostname 'smithi094' 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b71ac0 0x562370b707d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc1b95c6740 4 client.24991 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 --> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] -- command(tid 0: {"prefix": "status"}) -- 0x562370a5c3a0 con 0x562370b71ac0 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.558+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b71ac0 0x562370b707d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1a6a00640 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b71ac0 0x562370b707d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff13c0f6d778042e server_cookie=2f37bce3abc5b7fe in_seq=0 out_seq=0 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc19d600640 10 client.24991 ms_handle_connect on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc19d600640 1 -- 172.21.15.94:0/1963931960 <== mds.0 v2:172.21.15.137:6838/3444396688 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562370a5c3a0 con 0x562370b71ac0 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc19d600640 10 client.24991 handle_command_reply: tid=0 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 client.24991 shutdown 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 msgr2=0x562370aff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370aff760 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1a002a9e0 tx=0x7fc1a00058f0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 shutdown_connections 2024-04-25T15:01:27.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x562370af5cb0 0x562370b6b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562370af0b40 0x562370b69170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x562370b71ac0 0x562370b707d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 --2- 172.21.15.94:0/1963931960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562370af0150 0x562370aff760 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc19d600640 0 client.24991 ms_handle_reset on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 >> 172.21.15.94:0/1963931960 conn(0x562370aeb7c0 msgr2=0x562370af1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 shutdown_connections 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.559+0000 7fc1b95c6740 1 -- 172.21.15.94:0/1963931960 wait complete. 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b95c6740 20 client.24991 trim_cache size 0 max 16384 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T15:01:27.565 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f04d929c-0312-11ef-bc93-c7b262605968", 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24596, 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 100.988471623, 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 19, 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 88, 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 107.440989605 2024-04-25T15:01:27.566 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1940a3c30 msgr2=0x7fc1940a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1940a3c30 0x7fc1940a60f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562370af13a0 tx=0x7fc198009850 comp rx=0 tx=0).stop 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01019d0 msgr2=0x7fc1b01a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01019d0 0x7fc1b01a17f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc1a800eb10 tx=0x7fc1a800c2d0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 shutdown_connections 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc1940a3c30 0x7fc1940a60f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc1b01019d0 0x7fc1b01a17f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1b0101030 0x7fc1b019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 --2- 172.21.15.94:0/1936835390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc1b01006d0 0x7fc1b019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 >> 172.21.15.94:0/1936835390 conn(0x7fc1b00fc2f0 msgr2=0x7fc1b00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.560+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 shutdown_connections 2024-04-25T15:01:27.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:27.561+0000 7fc1b5200640 1 -- 172.21.15.94:0/1936835390 wait complete. 2024-04-25T15:01:27.579 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T15:01:27.580 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:01:27.591 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:01:27.592 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T15:01:27.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:27 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3292760544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:27.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:27 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2872439060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:27.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:27 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3292760544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:27.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:27 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2872439060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:27 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3292760544' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:27.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:27 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2872439060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:28.033 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:28.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:28 smithi138 ceph-mon[32875]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:28 smithi094 ceph-mon[27862]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:28 smithi137 ceph-mon[32439]: pgmap v361: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:30.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.369+0000 7fd886e00700 1 -- 172.21.15.94:0/1065486394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880102ab0 msgr2=0x7fd880102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.369+0000 7fd886e00700 1 --2- 172.21.15.94:0/1065486394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880102ab0 0x7fd880102ed0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b0d0 tx=0x7fd87000b3e0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.370+0000 7fd886e00700 1 -- 172.21.15.94:0/1065486394 shutdown_connections 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.370+0000 7fd886e00700 1 --2- 172.21.15.94:0/1065486394 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880104650 0x7fd88010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.370+0000 7fd886e00700 1 --2- 172.21.15.94:0/1065486394 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880102ab0 0x7fd880102ed0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.370+0000 7fd886e00700 1 --2- 172.21.15.94:0/1065486394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880103cb0 0x7fd880104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.370+0000 7fd886e00700 1 -- 172.21.15.94:0/1065486394 >> 172.21.15.94:0/1065486394 conn(0x7fd8800fe010 msgr2=0x7fd880100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.370+0000 7fd886e00700 1 -- 172.21.15.94:0/1065486394 shutdown_connections 2024-04-25T15:01:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.370+0000 7fd886e00700 1 -- 172.21.15.94:0/1065486394 wait complete. 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.371+0000 7fd886e00700 1 Processor -- start 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.371+0000 7fd886e00700 1 -- start start 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd886e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102ab0 0x7fd88019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd886e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880103cb0 0x7fd88019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd886e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880104650 0x7fd8801a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd886e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd88010e520 con 0x7fd880102ab0 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd886e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd88010e3a0 con 0x7fd880104650 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd886e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd88010e6a0 con 0x7fd880103cb0 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd87fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102ab0 0x7fd88019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd87fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102ab0 0x7fd88019ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34158/0 (socket says 172.21.15.94:34158) 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd87fe00700 1 -- 172.21.15.94:0/1500477506 learned_addr learned my addr 172.21.15.94:0/1500477506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd877400700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880103cb0 0x7fd88019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd884e00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880104650 0x7fd8801a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd87fe00700 1 -- 172.21.15.94:0/1500477506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880103cb0 msgr2=0x7fd88019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd87fe00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880103cb0 0x7fd88019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd87fe00700 1 -- 172.21.15.94:0/1500477506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880104650 msgr2=0x7fd8801a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.372+0000 7fd87fe00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880104650 0x7fd8801a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.373+0000 7fd87fe00700 1 -- 172.21.15.94:0/1500477506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8801a2a60 con 0x7fd880102ab0 2024-04-25T15:01:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.373+0000 7fd877400700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880103cb0 0x7fd88019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.373+0000 7fd87fe00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102ab0 0x7fd88019ae00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b770 tx=0x7fd870004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.373+0000 7fd87d600700 1 -- 172.21.15.94:0/1500477506 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87001f070 con 0x7fd880102ab0 2024-04-25T15:01:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.373+0000 7fd87d600700 1 -- 172.21.15.94:0/1500477506 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd870007b80 con 0x7fd880102ab0 2024-04-25T15:01:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.373+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8801a2cf0 con 0x7fd880102ab0 2024-04-25T15:01:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.373+0000 7fd87d600700 1 -- 172.21.15.94:0/1500477506 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87001b690 con 0x7fd880102ab0 2024-04-25T15:01:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.374+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801a31a0 con 0x7fd880102ab0 2024-04-25T15:01:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.375+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd880069730 con 0x7fd880102ab0 2024-04-25T15:01:30.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.377+0000 7fd87d600700 1 -- 172.21.15.94:0/1500477506 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd870022070 con 0x7fd880102ab0 2024-04-25T15:01:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.378+0000 7fd87d600700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8640a3cc0 0x7fd8640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.378+0000 7fd87d600700 1 -- 172.21.15.94:0/1500477506 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd87000f020 con 0x7fd880102ab0 2024-04-25T15:01:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.378+0000 7fd877400700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8640a3cc0 0x7fd8640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.379+0000 7fd877400700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8640a3cc0 0x7fd8640a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd868009680 tx=0x7fd868009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.382+0000 7fd87d600700 1 -- 172.21.15.94:0/1500477506 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd870029020 con 0x7fd880102ab0 2024-04-25T15:01:30.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.632+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd880076390 con 0x7fd8640a3cc0 2024-04-25T15:01:30.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.634+0000 7fd87d600700 1 -- 172.21.15.94:0/1500477506 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd880076390 con 0x7fd8640a3cc0 2024-04-25T15:01:30.636 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-04-25T15:01:30.636 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-04-25T15:01:30.636 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8640a3cc0 msgr2=0x7fd8640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8640a3cc0 0x7fd8640a6180 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd868009680 tx=0x7fd868009260 comp rx=0 tx=0).stop 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102ab0 msgr2=0x7fd88019ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102ab0 0x7fd88019ae00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b770 tx=0x7fd870004ce0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 shutdown_connections 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8640a3cc0 0x7fd8640a6180 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd880104650 0x7fd8801a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd880102ab0 0x7fd88019ae00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 --2- 172.21.15.94:0/1500477506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd880103cb0 0x7fd88019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:30.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 >> 172.21.15.94:0/1500477506 conn(0x7fd8800fe010 msgr2=0x7fd880074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:30.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.637+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 shutdown_connections 2024-04-25T15:01:30.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:30.638+0000 7fd886e00700 1 -- 172.21.15.94:0/1500477506 wait complete. 2024-04-25T15:01:30.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:30 smithi138 ceph-mon[32875]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:30 smithi094 ceph-mon[27862]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:30 smithi137 ceph-mon[32439]: pgmap v362: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:31.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T15:01:31.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:31 smithi094 ceph-mon[27862]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:31.925 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:31.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:31 smithi138 ceph-mon[32875]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:31.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:31 smithi137 ceph-mon[32439]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:32.918 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:32 smithi094 ceph-mon[27862]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:32.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:32 smithi138 ceph-mon[32875]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:32.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:32 smithi137 ceph-mon[32439]: pgmap v363: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:34.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.150+0000 7f8502800700 1 -- 172.21.15.94:0/4196896098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10a2e0 msgr2=0x7f84fc10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:34.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.150+0000 7f8502800700 1 --2- 172.21.15.94:0/4196896098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10a2e0 0x7f84fc10a740 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f84e80099c0 tx=0x7f84e8009cd0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.150+0000 7f8502800700 1 -- 172.21.15.94:0/4196896098 shutdown_connections 2024-04-25T15:01:34.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.150+0000 7f8502800700 1 --2- 172.21.15.94:0/4196896098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84fc10ac80 0x7f84fc111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.150+0000 7f8502800700 1 --2- 172.21.15.94:0/4196896098 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84fc1090e0 0x7f84fc109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.150+0000 7f8502800700 1 --2- 172.21.15.94:0/4196896098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10a2e0 0x7f84fc10a740 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.150+0000 7f8502800700 1 -- 172.21.15.94:0/4196896098 >> 172.21.15.94:0/4196896098 conn(0x7f84fc0780b0 msgr2=0x7f84fc07a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:34.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.151+0000 7f8502800700 1 -- 172.21.15.94:0/4196896098 shutdown_connections 2024-04-25T15:01:34.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.151+0000 7f8502800700 1 -- 172.21.15.94:0/4196896098 wait complete. 2024-04-25T15:01:34.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.152+0000 7f8502800700 1 Processor -- start 2024-04-25T15:01:34.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.152+0000 7f8502800700 1 -- start start 2024-04-25T15:01:34.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.152+0000 7f8502800700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84fc1090e0 0x7f84fc19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:34.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.152+0000 7f8502800700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84fc10a2e0 0x7f84fc19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:34.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.152+0000 7f8502800700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10ac80 0x7f84fc1a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:34.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f8502800700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc114bb0 con 0x7f84fc10ac80 2024-04-25T15:01:34.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f8502800700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc114a30 con 0x7f84fc1090e0 2024-04-25T15:01:34.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f8502800700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc114d30 con 0x7f84fc10a2e0 2024-04-25T15:01:34.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10ac80 0x7f84fc1a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:34.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10ac80 0x7f84fc1a67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34182/0 (socket says 172.21.15.94:34182) 2024-04-25T15:01:34.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 -- 172.21.15.94:0/2178166721 learned_addr learned my addr 172.21.15.94:0/2178166721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:34.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fb400700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84fc1090e0 0x7f84fc19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:34.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84faa00700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84fc10a2e0 0x7f84fc19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 -- 172.21.15.94:0/2178166721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84fc10a2e0 msgr2=0x7f84fc19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84fc10a2e0 0x7f84fc19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 -- 172.21.15.94:0/2178166721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84fc1090e0 msgr2=0x7f84fc19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84fc1090e0 0x7f84fc19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fbe00700 1 -- 172.21.15.94:0/2178166721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc1a6e50 con 0x7f84fc10ac80 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.153+0000 7f84fb400700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84fc1090e0 0x7f84fc19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.154+0000 7f84fbe00700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10ac80 0x7f84fc1a67b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84f400ef60 tx=0x7f84f400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.154+0000 7f84f3e00700 1 -- 172.21.15.94:0/2178166721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f40092e0 con 0x7f84fc10ac80 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.154+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84fc1a70e0 con 0x7f84fc10ac80 2024-04-25T15:01:34.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.154+0000 7f84f3e00700 1 -- 172.21.15.94:0/2178166721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84f4004bb0 con 0x7f84fc10ac80 2024-04-25T15:01:34.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.154+0000 7f84f3e00700 1 -- 172.21.15.94:0/2178166721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4004020 con 0x7f84fc10ac80 2024-04-25T15:01:34.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.154+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84fc1a7620 con 0x7f84fc10ac80 2024-04-25T15:01:34.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.155+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84fc069730 con 0x7f84fc10ac80 2024-04-25T15:01:34.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.157+0000 7f84f3e00700 1 -- 172.21.15.94:0/2178166721 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f84f401b030 con 0x7f84fc10ac80 2024-04-25T15:01:34.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.158+0000 7f84f3e00700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f84e00a3c40 0x7f84e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:34.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.159+0000 7f84f3e00700 1 -- 172.21.15.94:0/2178166721 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f84f40cfa70 con 0x7f84fc10ac80 2024-04-25T15:01:34.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.159+0000 7f84fb400700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f84e00a3c40 0x7f84e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:34.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.159+0000 7f84fb400700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f84e00a3c40 0x7f84e00a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84fc10a140 tx=0x7f84ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:34.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.163+0000 7f84f3e00700 1 -- 172.21.15.94:0/2178166721 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84f4010040 con 0x7f84fc10ac80 2024-04-25T15:01:34.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.395+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f84fc062f40 con 0x7f84e00a3c40 2024-04-25T15:01:34.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.435+0000 7f84f3e00700 1 -- 172.21.15.94:0/2178166721 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f84fc062f40 con 0x7f84e00a3c40 2024-04-25T15:01:34.437 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:01:34.437 INFO:teuthology.orchestra.run.smithi094.stdout:mds.a smithi094 running (117s) 23s ago 117s 12.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8c712da21519 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.b smithi137 running (114s) 45s ago 114s 25.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 0c3ed092aa3c 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.c smithi138 running (2m) 46s ago 2m 13.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b76cdd18b0ed 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.d smithi094 running (2m) 23s ago 2m 12.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c99bf4f29c5a 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.e smithi137 running (2m) 45s ago 2m 13.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 513567bf6136 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.f smithi138 running (2m) 46s ago 2m 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c20d22a287a8 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.g smithi094 running (2m) 23s ago 2m 17.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 48077639c94e 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.h smithi137 running (2m) 45s ago 2m 16.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4976fcf0fff1 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.i smithi138 running (2m) 46s ago 2m 15.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 84824fb92537 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.j smithi094 running (2m) 23s ago 2m 12.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7579c7a12259 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.k smithi137 running (2m) 45s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a93f09d911cc 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mds.l smithi138 running (2m) 46s ago 2m 14.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e7ad3ca004a5 2024-04-25T15:01:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.x smithi094 *:9283,8765 running (10m) 23s ago 10m 513M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2441008644f0 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.y smithi137 *:8443,8765 running (8m) 45s ago 8m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 10124ef5b4b5 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.z smithi138 *:8443,8765 running (8m) 46s ago 8m 451M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2a448e3332f9 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a smithi094 running (10m) 23s ago 10m 50.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 f57cc4dcd35b 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b smithi137 running (8m) 45s ago 8m 43.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 af3d393348f4 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c smithi138 running (8m) 46s ago 8m 43.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 ef5fb5e28152 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 23s ago 7m 49.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 04e9af904d9d 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi137 running (7m) 45s ago 7m 56.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 5b5c5406973c 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi138 running (6m) 46s ago 6m 59.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 d403b70cd4e0 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi094 running (6m) 23s ago 6m 46.6M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ea9503ecc2ed 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi137 running (6m) 45s ago 6m 52.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 e8149acb951f 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi138 running (5m) 46s ago 5m 46.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ee78f9788bb2 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 smithi094 running (5m) 23s ago 5m 49.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 739d48971d5d 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 smithi137 running (4m) 45s ago 4m 45.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 dfcf59ca46ed 2024-04-25T15:01:34.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 smithi138 running (4m) 46s ago 4m 48.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 fc91bdf21567 2024-04-25T15:01:34.440 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 smithi094 running (3m) 23s ago 3m 48.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c33294b126e6 2024-04-25T15:01:34.440 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 smithi137 running (3m) 45s ago 3m 48.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 43a0f1489bfd 2024-04-25T15:01:34.440 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 smithi138 running (2m) 46s ago 2m 44.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 c8025cfa318c 2024-04-25T15:01:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f84e00a3c40 msgr2=0x7f84e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f84e00a3c40 0x7f84e00a6100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84fc10a140 tx=0x7f84ec009260 comp rx=0 tx=0).stop 2024-04-25T15:01:34.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10ac80 msgr2=0x7f84fc1a67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10ac80 0x7f84fc1a67b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f84f400ef60 tx=0x7f84f400ef90 comp rx=0 tx=0).stop 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 shutdown_connections 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f84e00a3c40 0x7f84e00a6100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84fc10ac80 0x7f84fc1a67b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84fc1090e0 0x7f84fc19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 --2- 172.21.15.94:0/2178166721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84fc10a2e0 0x7f84fc19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.438+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 >> 172.21.15.94:0/2178166721 conn(0x7f84fc0780b0 msgr2=0x7f84fc10d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.439+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 shutdown_connections 2024-04-25T15:01:34.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:34.439+0000 7f8502800700 1 -- 172.21.15.94:0/2178166721 wait complete. 2024-04-25T15:01:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:34 smithi138 ceph-mon[32875]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:34.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:34 smithi094 ceph-mon[27862]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:34 smithi137 ceph-mon[32439]: pgmap v364: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:35.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T15:01:35.699 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:35.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:35 smithi138 ceph-mon[32875]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:35.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:35 smithi094 ceph-mon[27862]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:35.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:35 smithi137 ceph-mon[32439]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:36 smithi138 ceph-mon[32875]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:36.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:36 smithi094 ceph-mon[27862]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:36 smithi137 ceph-mon[32439]: pgmap v365: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.875+0000 7fd822200700 1 -- 172.21.15.94:0/752439792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c104650 msgr2=0x7fd81c10af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.875+0000 7fd822200700 1 --2- 172.21.15.94:0/752439792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c104650 0x7fd81c10af00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8140099c0 tx=0x7fd814009cd0 comp rx=0 tx=0).stop 2024-04-25T15:01:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.876+0000 7fd822200700 1 -- 172.21.15.94:0/752439792 shutdown_connections 2024-04-25T15:01:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.876+0000 7fd822200700 1 --2- 172.21.15.94:0/752439792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c104650 0x7fd81c10af00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.876+0000 7fd822200700 1 --2- 172.21.15.94:0/752439792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c102ab0 0x7fd81c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.876+0000 7fd822200700 1 --2- 172.21.15.94:0/752439792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd81c103cb0 0x7fd81c104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.876+0000 7fd822200700 1 -- 172.21.15.94:0/752439792 >> 172.21.15.94:0/752439792 conn(0x7fd81c0fe010 msgr2=0x7fd81c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.876+0000 7fd822200700 1 -- 172.21.15.94:0/752439792 shutdown_connections 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.876+0000 7fd822200700 1 -- 172.21.15.94:0/752439792 wait complete. 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 Processor -- start 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 -- start start 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c102ab0 0x7fd81c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c103cb0 0x7fd81c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd81c104650 0x7fd81c1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c10e580 con 0x7fd81c102ab0 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c10e400 con 0x7fd81c104650 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd822200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c10e700 con 0x7fd81c103cb0 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd81b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c102ab0 0x7fd81c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd81b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c102ab0 0x7fd81c198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52002/0 (socket says 172.21.15.94:52002) 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd81b400700 1 -- 172.21.15.94:0/4054554952 learned_addr learned my addr 172.21.15.94:0/4054554952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.877+0000 7fd81be00700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd81c104650 0x7fd81c1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.878+0000 7fd812a00700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c103cb0 0x7fd81c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.878+0000 7fd81b400700 1 -- 172.21.15.94:0/4054554952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c103cb0 msgr2=0x7fd81c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.878+0000 7fd81b400700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c103cb0 0x7fd81c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.878+0000 7fd81b400700 1 -- 172.21.15.94:0/4054554952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd81c104650 msgr2=0x7fd81c1a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.878+0000 7fd81b400700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd81c104650 0x7fd81c1a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:37.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.878+0000 7fd81b400700 1 -- 172.21.15.94:0/4054554952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c1a0830 con 0x7fd81c102ab0 2024-04-25T15:01:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.878+0000 7fd81b400700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c102ab0 0x7fd81c198c20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00e9a0 tx=0x7fd80c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.879+0000 7fd818c00700 1 -- 172.21.15.94:0/4054554952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c004220 con 0x7fd81c102ab0 2024-04-25T15:01:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.879+0000 7fd818c00700 1 -- 172.21.15.94:0/4054554952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd80c0043c0 con 0x7fd81c102ab0 2024-04-25T15:01:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.879+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c1a0b20 con 0x7fd81c102ab0 2024-04-25T15:01:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.879+0000 7fd818c00700 1 -- 172.21.15.94:0/4054554952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c003cc0 con 0x7fd81c102ab0 2024-04-25T15:01:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.879+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c1a10b0 con 0x7fd81c102ab0 2024-04-25T15:01:37.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.880+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c069730 con 0x7fd81c102ab0 2024-04-25T15:01:37.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.882+0000 7fd818c00700 1 -- 172.21.15.94:0/4054554952 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd80c00c810 con 0x7fd81c102ab0 2024-04-25T15:01:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.884+0000 7fd818c00700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8000a3cc0 0x7fd8000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:37.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.884+0000 7fd818c00700 1 -- 172.21.15.94:0/4054554952 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd80c0d0590 con 0x7fd81c102ab0 2024-04-25T15:01:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.884+0000 7fd812a00700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8000a3cc0 0x7fd8000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:37.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.885+0000 7fd812a00700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8000a3cc0 0x7fd8000a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd804009da0 tx=0x7fd80400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:37.888+0000 7fd818c00700 1 -- 172.21.15.94:0/4054554952 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd80c09cf30 con 0x7fd81c102ab0 2024-04-25T15:01:38.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.126+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fd81c062e80 con 0x7fd8000a3cc0 2024-04-25T15:01:38.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.135+0000 7fd818c00700 1 -- 172.21.15.94:0/4054554952 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fd81c062e80 con 0x7fd8000a3cc0 2024-04-25T15:01:38.137 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T15:01:38.137 INFO:teuthology.orchestra.run.smithi094.stdout:mds.all 12/12 49s ago 2m smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi137=b;smithi137=e;smithi137=h;smithi137=k;smithi138=c;smithi138=f;smithi138=i;smithi138=l;count:12 2024-04-25T15:01:38.137 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 3/3 49s ago 8m smithi094=x;smithi137=y;smithi138=z;count:3 2024-04-25T15:01:38.137 INFO:teuthology.orchestra.run.smithi094.stdout:mon 3/3 49s ago 8m smithi094:172.21.15.94=a;smithi137:172.21.15.137=b;smithi138:172.21.15.138=c;count:3 2024-04-25T15:01:38.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd 12 49s ago - 2024-04-25T15:01:38.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8000a3cc0 msgr2=0x7fd8000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:38.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8000a3cc0 0x7fd8000a6180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd804009da0 tx=0x7fd80400b040 comp rx=0 tx=0).stop 2024-04-25T15:01:38.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c102ab0 msgr2=0x7fd81c198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:38.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c102ab0 0x7fd81c198c20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00e9a0 tx=0x7fd80c00ecb0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 shutdown_connections 2024-04-25T15:01:38.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd8000a3cc0 0x7fd8000a6180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd81c104650 0x7fd81c1a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd81c102ab0 0x7fd81c198c20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 --2- 172.21.15.94:0/4054554952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c103cb0 0x7fd81c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.138+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 >> 172.21.15.94:0/4054554952 conn(0x7fd81c0fe010 msgr2=0x7fd81c106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.139+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 shutdown_connections 2024-04-25T15:01:38.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:38.139+0000 7fd822200700 1 -- 172.21.15.94:0/4054554952 wait complete. 2024-04-25T15:01:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:38 smithi138 ceph-mon[32875]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:38 smithi094 ceph-mon[27862]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:38 smithi137 ceph-mon[32439]: pgmap v366: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:39.056 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T15:01:39.507 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:39.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:39 smithi138 ceph-mon[32875]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:39.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:39 smithi094 ceph-mon[27862]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:39.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:39 smithi137 ceph-mon[32439]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:40.907 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:40 smithi094 ceph-mon[27862]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:40 smithi138 ceph-mon[32875]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:40 smithi137 ceph-mon[32439]: pgmap v367: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:41.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.734+0000 7fc39b400700 1 -- 172.21.15.94:0/3672229323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39410f090 msgr2=0x7fc394111480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.734+0000 7fc39b400700 1 --2- 172.21.15.94:0/3672229323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39410f090 0x7fc394111480 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009f90 tx=0x7fc38c00b330 comp rx=0 tx=0).stop 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.734+0000 7fc39b400700 1 -- 172.21.15.94:0/3672229323 shutdown_connections 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.734+0000 7fc39b400700 1 --2- 172.21.15.94:0/3672229323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc39410f090 0x7fc394111480 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.734+0000 7fc39b400700 1 --2- 172.21.15.94:0/3672229323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc394076a70 0x7fc394074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.734+0000 7fc39b400700 1 --2- 172.21.15.94:0/3672229323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 0x7fc394075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.734+0000 7fc39b400700 1 -- 172.21.15.94:0/3672229323 >> 172.21.15.94:0/3672229323 conn(0x7fc3940fe010 msgr2=0x7fc394100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.735+0000 7fc39b400700 1 -- 172.21.15.94:0/3672229323 shutdown_connections 2024-04-25T15:01:41.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.735+0000 7fc39b400700 1 -- 172.21.15.94:0/3672229323 wait complete. 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 Processor -- start 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 -- start start 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 0x7fc39419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc394076a70 0x7fc39419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc39410f090 0x7fc3941a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fc394114b90 con 0x7fc3940754a0 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fc394114a10 con 0x7fc394076a70 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.736+0000 7fc39b400700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc394114d10 con 0x7fc39410f090 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc398a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 0x7fc39419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:41.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc393e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc394076a70 0x7fc39419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc398a00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 0x7fc39419f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52026/0 (socket says 172.21.15.94:52026) 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc398a00700 1 -- 172.21.15.94:0/1529437588 learned_addr learned my addr 172.21.15.94:0/1529437588 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc399400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc39410f090 0x7fc3941a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc393e00700 1 -- 172.21.15.94:0/1529437588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc39410f090 msgr2=0x7fc3941a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc393e00700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc39410f090 0x7fc3941a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc393e00700 1 -- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 msgr2=0x7fc39419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc393e00700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 0x7fc39419f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc393e00700 1 -- 172.21.15.94:0/1529437588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3941a6e10 con 0x7fc394076a70 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc399400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc39410f090 0x7fc3941a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.737+0000 7fc398a00700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 0x7fc39419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:41.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.738+0000 7fc393e00700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc394076a70 0x7fc39419f6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc38000e9c0 tx=0x7fc38000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.738+0000 7fc391600700 1 -- 172.21.15.94:0/1529437588 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380004220 con 0x7fc394076a70 2024-04-25T15:01:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.738+0000 7fc391600700 1 -- 172.21.15.94:0/1529437588 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3800043c0 con 0x7fc394076a70 2024-04-25T15:01:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.738+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3941a7100 con 0x7fc394076a70 2024-04-25T15:01:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.738+0000 7fc391600700 1 -- 172.21.15.94:0/1529437588 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380003c30 con 0x7fc394076a70 2024-04-25T15:01:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.738+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3941a7690 con 0x7fc394076a70 2024-04-25T15:01:41.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.739+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc394069730 con 0x7fc394076a70 2024-04-25T15:01:41.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.741+0000 7fc391600700 1 -- 172.21.15.94:0/1529437588 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc38000c7c0 con 0x7fc394076a70 2024-04-25T15:01:41.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.743+0000 7fc391600700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc37c0a3d40 0x7fc37c0a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:41.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.743+0000 7fc391600700 1 -- 172.21.15.94:0/1529437588 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fc3800d0590 con 0x7fc394076a70 2024-04-25T15:01:41.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.743+0000 7fc398a00700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc37c0a3d40 0x7fc37c0a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:41.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.744+0000 7fc398a00700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc37c0a3d40 0x7fc37c0a6200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc388009750 tx=0x7fc388009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:41.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.747+0000 7fc391600700 1 -- 172.21.15.94:0/1529437588 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc38009cf30 con 0x7fc394076a70 2024-04-25T15:01:41.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.983+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc394062e80 con 0x7fc37c0a3d40 2024-04-25T15:01:41.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.987+0000 7fc391600700 1 -- 172.21.15.94:0/1529437588 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc394062e80 con 0x7fc37c0a3d40 2024-04-25T15:01:41.987 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-04-25T15:01:41.987 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-04-25T15:01:41.987 INFO:teuthology.orchestra.run.smithi094.stdout:smithi137 172.21.15.137 2024-04-25T15:01:41.987 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 172.21.15.138 2024-04-25T15:01:41.988 INFO:teuthology.orchestra.run.smithi094.stdout:3 hosts in cluster 2024-04-25T15:01:41.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.989+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc37c0a3d40 msgr2=0x7fc37c0a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:41.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.989+0000 7fc39b400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc37c0a3d40 0x7fc37c0a6200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc388009750 tx=0x7fc388009310 comp rx=0 tx=0).stop 2024-04-25T15:01:41.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.989+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc394076a70 msgr2=0x7fc39419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:41.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.989+0000 7fc39b400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc394076a70 0x7fc39419f6f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc38000e9c0 tx=0x7fc38000ecd0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 shutdown_connections 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fc37c0a3d40 0x7fc37c0a6200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc39410f090 0x7fc3941a6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3940754a0 0x7fc39419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 --2- 172.21.15.94:0/1529437588 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fc394076a70 0x7fc39419f6f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 >> 172.21.15.94:0/1529437588 conn(0x7fc3940fe010 msgr2=0x7fc394102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 shutdown_connections 2024-04-25T15:01:41.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:41.990+0000 7fc39b400700 1 -- 172.21.15.94:0/1529437588 wait complete. 2024-04-25T15:01:42.879 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T15:01:42.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:42 smithi094 ceph-mon[27862]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:42.956 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:42 smithi138 ceph-mon[32875]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:42 smithi137 ceph-mon[32439]: pgmap v368: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:43.335 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:43.794 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:43 smithi094 ceph-mon[27862]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:43.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:43 smithi138 ceph-mon[32875]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:43.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:43 smithi137 ceph-mon[32439]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:44 smithi138 ceph-mon[32875]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:44 smithi094 ceph-mon[27862]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:44.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:44 smithi137 ceph-mon[32439]: pgmap v369: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:45.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 -- 172.21.15.94:0/1307453792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d810a2e0 msgr2=0x7f65d810a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 --2- 172.21.15.94:0/1307453792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d810a2e0 0x7f65d810a740 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f65c40099c0 tx=0x7f65c4009cd0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 -- 172.21.15.94:0/1307453792 shutdown_connections 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 --2- 172.21.15.94:0/1307453792 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810ac80 0x7f65d8111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 --2- 172.21.15.94:0/1307453792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65d81090e0 0x7f65d8109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 --2- 172.21.15.94:0/1307453792 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d810a2e0 0x7f65d810a740 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 -- 172.21.15.94:0/1307453792 >> 172.21.15.94:0/1307453792 conn(0x7f65d80780b0 msgr2=0x7f65d807a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 -- 172.21.15.94:0/1307453792 shutdown_connections 2024-04-25T15:01:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.402+0000 7f65dd200700 1 -- 172.21.15.94:0/1307453792 wait complete. 2024-04-25T15:01:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 Processor -- start 2024-04-25T15:01:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 -- start start 2024-04-25T15:01:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d81090e0 0x7f65d819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810a2e0 0x7f65d819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65d810ac80 0x7f65d81a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8114bb0 con 0x7f65d810a2e0 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8114a30 con 0x7f65d81090e0 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.404+0000 7f65dd200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8114d30 con 0x7f65d810ac80 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810a2e0 0x7f65d819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d81090e0 0x7f65d819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5400700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810a2e0 0x7f65d819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52036/0 (socket says 172.21.15.94:52036) 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5400700 1 -- 172.21.15.94:0/2680204146 learned_addr learned my addr 172.21.15.94:0/2680204146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d6800700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65d810ac80 0x7f65d81a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5e00700 1 -- 172.21.15.94:0/2680204146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65d810ac80 msgr2=0x7f65d81a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5e00700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65d810ac80 0x7f65d81a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5e00700 1 -- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810a2e0 msgr2=0x7f65d819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5e00700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810a2e0 0x7f65d819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5e00700 1 -- 172.21.15.94:0/2680204146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d81a6e30 con 0x7f65d81090e0 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d6800700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65d810ac80 0x7f65d81a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.405+0000 7f65d5400700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810a2e0 0x7f65d819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.406+0000 7f65d5e00700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d81090e0 0x7f65d819f1d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f65c800e9c0 tx=0x7f65c800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.406+0000 7f65cea00700 1 -- 172.21.15.94:0/2680204146 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8004220 con 0x7f65d81090e0 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.406+0000 7f65cea00700 1 -- 172.21.15.94:0/2680204146 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c80043c0 con 0x7f65d81090e0 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.406+0000 7f65cea00700 1 -- 172.21.15.94:0/2680204146 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c8003c30 con 0x7f65d81090e0 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.406+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d81a7120 con 0x7f65d81090e0 2024-04-25T15:01:45.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.406+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a7610 con 0x7f65d81090e0 2024-04-25T15:01:45.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.407+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d8069730 con 0x7f65d81090e0 2024-04-25T15:01:45.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.410+0000 7f65cea00700 1 -- 172.21.15.94:0/2680204146 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f65c800c7c0 con 0x7f65d81090e0 2024-04-25T15:01:45.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.411+0000 7f65cea00700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65bc0a3cc0 0x7f65bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:45.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.411+0000 7f65cea00700 1 -- 172.21.15.94:0/2680204146 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f65c80d06b0 con 0x7f65d81090e0 2024-04-25T15:01:45.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.411+0000 7f65d5400700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65bc0a3cc0 0x7f65bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:45.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.412+0000 7f65d5400700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65bc0a3cc0 0x7f65bc0a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f65c4000c00 tx=0x7f65c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.413+0000 7f65cea00700 1 -- 172.21.15.94:0/2680204146 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65c809d0e0 con 0x7f65d81090e0 2024-04-25T15:01:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.646+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f65d8062e80 con 0x7f65bc0a3cc0 2024-04-25T15:01:45.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.650+0000 7f65cea00700 1 -- 172.21.15.94:0/2680204146 <== mgr.14152 v2:172.21.15.94:6800/2485357518 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f65d8062e80 con 0x7f65bc0a3cc0 2024-04-25T15:01:45.651 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T15:01:45.651 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T15:01:45.651 INFO:teuthology.orchestra.run.smithi094.stdout:smithi137 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233004A400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T15:01:45.651 INFO:teuthology.orchestra.run.smithi094.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T15:01:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.653+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65bc0a3cc0 msgr2=0x7f65bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.653+0000 7f65dd200700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65bc0a3cc0 0x7f65bc0a6180 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f65c4000c00 tx=0x7f65c4005fd0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.653+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d81090e0 msgr2=0x7f65d819f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.653+0000 7f65dd200700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d81090e0 0x7f65d819f1d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f65c800e9c0 tx=0x7f65c800ecd0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 shutdown_connections 2024-04-25T15:01:45.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f65bc0a3cc0 0x7f65bc0a6180 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65d810ac80 0x7f65d81a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f65d81090e0 0x7f65d819f1d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 --2- 172.21.15.94:0/2680204146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f65d810a2e0 0x7f65d819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:45.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 >> 172.21.15.94:0/2680204146 conn(0x7f65d80780b0 msgr2=0x7f65d810d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:45.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 shutdown_connections 2024-04-25T15:01:45.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:45.654+0000 7f65dd200700 1 -- 172.21.15.94:0/2680204146 wait complete. 2024-04-25T15:01:46.530 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:01:46.546 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:01:46.546 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T15:01:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:46 smithi138 ceph-mon[32875]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:46 smithi138 ceph-mon[32875]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:46.961 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:46 smithi094 ceph-mon[27862]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:46.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:46 smithi094 ceph-mon[27862]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:46 smithi137 ceph-mon[32439]: pgmap v370: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:01:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:46 smithi137 ceph-mon[32439]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:01:46.994 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:48.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:48 smithi094 ceph-mon[27862]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:48 smithi138 ceph-mon[32875]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:48 smithi137 ceph-mon[32439]: pgmap v371: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:49.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.110+0000 7f135ae00700 1 -- 172.21.15.94:0/2704611517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102640 msgr2=0x7f1354102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:49.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.110+0000 7f135ae00700 1 --2- 172.21.15.94:0/2704611517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102640 0x7f1354102ac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f134c00b0a0 tx=0x7f134c00b3b0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.111+0000 7f135ae00700 1 -- 172.21.15.94:0/2704611517 shutdown_connections 2024-04-25T15:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.111+0000 7f135ae00700 1 --2- 172.21.15.94:0/2704611517 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354102640 0x7f1354102ac0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.111+0000 7f135ae00700 1 --2- 172.21.15.94:0/2704611517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 0x7f1354101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.111+0000 7f135ae00700 1 --2- 172.21.15.94:0/2704611517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354101c80 0x7f1354102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.111+0000 7f135ae00700 1 -- 172.21.15.94:0/2704611517 >> 172.21.15.94:0/2704611517 conn(0x7f13540fcf40 msgr2=0x7f13540ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.111+0000 7f135ae00700 1 -- 172.21.15.94:0/2704611517 shutdown_connections 2024-04-25T15:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.111+0000 7f135ae00700 1 -- 172.21.15.94:0/2704611517 wait complete. 2024-04-25T15:01:49.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 Processor -- start 2024-04-25T15:01:49.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 -- start start 2024-04-25T15:01:49.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 0x7f135419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:49.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354101c80 0x7f135419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354102640 0x7f13541a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f1354110770 con 0x7f1354101320 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f13541105f0 con 0x7f1354101c80 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.112+0000 7f135ae00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13541108f0 con 0x7f1354102640 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1353e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 0x7f135419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1353e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 0x7f135419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58164/0 (socket says 172.21.15.94:58164) 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1353e00700 1 -- 172.21.15.94:0/2965895585 learned_addr learned my addr 172.21.15.94:0/2965895585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1358e00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354102640 0x7f13541a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f134b400700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354101c80 0x7f135419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1358e00700 1 -- 172.21.15.94:0/2965895585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354101c80 msgr2=0x7f135419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1358e00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354101c80 0x7f135419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1358e00700 1 -- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 msgr2=0x7f135419ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1358e00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 0x7f135419ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1358e00700 1 -- 172.21.15.94:0/2965895585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13541a2a70 con 0x7f1354102640 2024-04-25T15:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1353e00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 0x7f135419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.113+0000 7f1358e00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354102640 0x7f13541a23d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f134c00b0a0 tx=0x7f134c004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.114+0000 7f1351600700 1 -- 172.21.15.94:0/2965895585 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c021070 con 0x7f1354102640 2024-04-25T15:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.114+0000 7f1351600700 1 -- 172.21.15.94:0/2965895585 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f134c007e30 con 0x7f1354102640 2024-04-25T15:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.114+0000 7f1351600700 1 -- 172.21.15.94:0/2965895585 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c004690 con 0x7f1354102640 2024-04-25T15:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.114+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13541a2d00 con 0x7f1354102640 2024-04-25T15:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.114+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13541a31b0 con 0x7f1354102640 2024-04-25T15:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.115+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1354069730 con 0x7f1354102640 2024-04-25T15:01:49.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.117+0000 7f1351600700 1 -- 172.21.15.94:0/2965895585 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f134c003d00 con 0x7f1354102640 2024-04-25T15:01:49.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.118+0000 7f1351600700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f13380a3cc0 0x7f13380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:49.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.119+0000 7f1351600700 1 -- 172.21.15.94:0/2965895585 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f134c0d14f0 con 0x7f1354102640 2024-04-25T15:01:49.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.119+0000 7f1353e00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f13380a3cc0 0x7f13380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:49.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.119+0000 7f1353e00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f13380a3cc0 0x7f13380a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1344011550 tx=0x7f1344009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:49.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.121+0000 7f1351600700 1 -- 172.21.15.94:0/2965895585 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f134c09df20 con 0x7f1354102640 2024-04-25T15:01:49.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.393+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f135402e040 con 0x7f1354102640 2024-04-25T15:01:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.394+0000 7f1351600700 1 -- 172.21.15.94:0/2965895585 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+3016 (secure 0 0 0) 0x7f134c09d820 con 0x7f1354102640 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:e19 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 19 2024-04-25T15:01:49.395 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-25T14:59:45.806095+0000 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-25T14:59:50.088429+0000 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 300 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 1 2024-04-25T15:01:49.396 INFO:teuthology.orchestra.run.smithi094.stdout:in 0 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24596} 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 0 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:qdb_cluster leader: 24596 members: 24596 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b{0:24596} state up:active seq 3 addr [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.g{0:24566} state up:standby-replay seq 1 addr [v2:172.21.15.94:6838/3586316280,v1:172.21.15.94:6839/3586316280] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.397 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.j{-1:14394} state up:standby seq 1 addr [v2:172.21.15.94:6834/3392009193,v1:172.21.15.94:6835/3392009193] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d{-1:14406} state up:standby seq 1 addr [v2:172.21.15.94:6836/1161222965,v1:172.21.15.94:6837/1161222965] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a{-1:14469} state up:standby seq 1 addr [v2:172.21.15.94:6840/2559487009,v1:172.21.15.94:6841/2559487009] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.l{-1:24454} state up:standby seq 1 addr [v2:172.21.15.138:6832/2043270325,v1:172.21.15.138:6833/2043270325] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.k{-1:24464} state up:standby seq 1 addr [v2:172.21.15.137:6832/2696996187,v1:172.21.15.137:6833/2696996187] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.e{-1:24506} state up:standby seq 1 addr [v2:172.21.15.137:6834/1346240831,v1:172.21.15.137:6835/1346240831] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.f{-1:24511} state up:standby seq 1 addr [v2:172.21.15.138:6834/4151262699,v1:172.21.15.138:6835/4151262699] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c{-1:24529} state up:standby seq 1 addr [v2:172.21.15.138:6836/4267251536,v1:172.21.15.138:6837/4267251536] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.h{-1:24542} state up:standby seq 1 addr [v2:172.21.15.137:6836/4197829661,v1:172.21.15.137:6837/4197829661] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.398 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.i{-1:24547} state up:standby seq 1 addr [v2:172.21.15.138:6838/593747621,v1:172.21.15.138:6839/593747621] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:01:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f13380a3cc0 msgr2=0x7f13380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f13380a3cc0 0x7f13380a6180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1344011550 tx=0x7f1344009260 comp rx=0 tx=0).stop 2024-04-25T15:01:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354102640 msgr2=0x7f13541a23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354102640 0x7f13541a23d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f134c00b0a0 tx=0x7f134c004c80 comp rx=0 tx=0).stop 2024-04-25T15:01:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 shutdown_connections 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f13380a3cc0 0x7f13380a6180 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1354102640 0x7f13541a23d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1354101320 0x7f135419ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 --2- 172.21.15.94:0/2965895585 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1354101c80 0x7f135419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 >> 172.21.15.94:0/2965895585 conn(0x7f13540fcf40 msgr2=0x7f13541002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 shutdown_connections 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:49.397+0000 7f135ae00700 1 -- 172.21.15.94:0/2965895585 wait complete. 2024-04-25T15:01:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:01:49.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:49 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2965895585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:01:49.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:49 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2965895585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:01:49.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:49 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2965895585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:01:50.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T15:01:50.694 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:01:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:50 smithi094 ceph-mon[27862]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:50 smithi137 ceph-mon[32439]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:51.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:50 smithi138 ceph-mon[32875]: pgmap v372: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:51.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:51 smithi094 ceph-mon[27862]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:52.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:51 smithi138 ceph-mon[32875]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:52.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:51 smithi137 ceph-mon[32439]: pgmap v373: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:52.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 -- 172.21.15.94:0/2930275763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f0010ac90 msgr2=0x7f7f00111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:52.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 --2- 172.21.15.94:0/2930275763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f0010ac90 0x7f7f00111540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800b0d0 tx=0x7f7ef800b3e0 comp rx=0 tx=0).stop 2024-04-25T15:01:52.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 -- 172.21.15.94:0/2930275763 shutdown_connections 2024-04-25T15:01:52.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 --2- 172.21.15.94:0/2930275763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f0010ac90 0x7f7f00111540 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:52.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 --2- 172.21.15.94:0/2930275763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f001090f0 0x7f7f00109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:52.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 --2- 172.21.15.94:0/2930275763 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f0010a2f0 0x7f7f0010a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:52.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 -- 172.21.15.94:0/2930275763 >> 172.21.15.94:0/2930275763 conn(0x7f7f000780c0 msgr2=0x7f7f0007a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:52.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 -- 172.21.15.94:0/2930275763 shutdown_connections 2024-04-25T15:01:52.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.774+0000 7f7f08600700 1 -- 172.21.15.94:0/2930275763 wait complete. 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.775+0000 7f7f08600700 1 Processor -- start 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f08600700 1 -- start start 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f08600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f001090f0 0x7f7f0019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f08600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f0010a2f0 0x7f7f0019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f08600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f0010ac90 0x7f7f001a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f08600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00114bf0 con 0x7f7f001090f0 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f08600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00114a70 con 0x7f7f0010a2f0 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f08600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f00114d70 con 0x7f7f0010ac90 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f05c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f001090f0 0x7f7f0019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:52.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f05c00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f001090f0 0x7f7f0019f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58170/0 (socket says 172.21.15.94:58170) 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f05c00700 1 -- 172.21.15.94:0/2963475149 learned_addr learned my addr 172.21.15.94:0/2963475149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.776+0000 7f7f05200700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f0010a2f0 0x7f7f0019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f06600700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f0010ac90 0x7f7f001a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f05c00700 1 -- 172.21.15.94:0/2963475149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f0010ac90 msgr2=0x7f7f001a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f05c00700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f0010ac90 0x7f7f001a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f05c00700 1 -- 172.21.15.94:0/2963475149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f0010a2f0 msgr2=0x7f7f0019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f05c00700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f0010a2f0 0x7f7f0019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f05c00700 1 -- 172.21.15.94:0/2963475149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f001a6e90 con 0x7f7f001090f0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f05c00700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f001090f0 0x7f7f0019f230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e980 tx=0x7f7ef400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7efe000700 1 -- 172.21.15.94:0/2963475149 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400cbd0 con 0x7f7f001090f0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7efe000700 1 -- 172.21.15.94:0/2963475149 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef40044e0 con 0x7f7f001090f0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f001a7180 con 0x7f7f001090f0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.777+0000 7f7efe000700 1 -- 172.21.15.94:0/2963475149 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4003e00 con 0x7f7f001090f0 2024-04-25T15:01:52.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.778+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f001a76c0 con 0x7f7f001090f0 2024-04-25T15:01:52.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.779+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f00069730 con 0x7f7f001090f0 2024-04-25T15:01:52.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.781+0000 7f7efe000700 1 -- 172.21.15.94:0/2963475149 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ef4003fa0 con 0x7f7f001090f0 2024-04-25T15:01:52.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.783+0000 7f7efe000700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7ee80a3cc0 0x7f7ee80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:52.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.783+0000 7f7efe000700 1 -- 172.21.15.94:0/2963475149 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f7ef40d0740 con 0x7f7f001090f0 2024-04-25T15:01:52.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.783+0000 7f7f05200700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7ee80a3cc0 0x7f7ee80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:52.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.784+0000 7f7f05200700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7ee80a3cc0 0x7f7ee80a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009680 tx=0x7f7eec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:52.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:52.788+0000 7f7efe000700 1 -- 172.21.15.94:0/2963475149 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ef409d0e0 con 0x7f7f001090f0 2024-04-25T15:01:53.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.015+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7f00063650 con 0x7f7f001090f0 2024-04-25T15:01:53.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.017+0000 7f7efe000700 1 -- 172.21.15.94:0/2963475149 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5302 (secure 0 0 0) 0x7f7ef409c9e0 con 0x7f7f001090f0 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 88 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:fsid f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-04-25T14:50:45.400368+0000 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-04-25T15:00:02.693179+0000 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:crush_version 26 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:full_ratio 0.95 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:backfillfull_ratio 0.9 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:nearfull_ratio 0.85 2024-04-25T15:01:53.019 INFO:teuthology.orchestra.run.smithi094.stdout:require_min_compat_client luminous 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:min_compat_client luminous 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:require_osd_release squid 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:stretch_mode_enabled false 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:max_osd 12 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6802/1239034236,v1:172.21.15.94:6803/1239034236] [v2:172.21.15.94:6804/1239034236,v1:172.21.15.94:6805/1239034236] exists,up a46fefd8-a368-4c2a-9038-54b8701077b7 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6800/3278622901,v1:172.21.15.137:6801/3278622901] [v2:172.21.15.137:6802/3278622901,v1:172.21.15.137:6803/3278622901] exists,up 385d47c9-0cd9-424b-b924-411ae615e8ba 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/3663095065,v1:172.21.15.138:6801/3663095065] [v2:172.21.15.138:6802/3663095065,v1:172.21.15.138:6803/3663095065] exists,up 3318c983-93b2-43b5-ad3b-a833ac35c895 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6810/462961415,v1:172.21.15.94:6811/462961415] [v2:172.21.15.94:6812/462961415,v1:172.21.15.94:6813/462961415] exists,up 0afc1aca-e4d4-405f-a3e6-f80504727eec 2024-04-25T15:01:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6808/2944995107,v1:172.21.15.137:6809/2944995107] [v2:172.21.15.137:6810/2944995107,v1:172.21.15.137:6811/2944995107] exists,up a8898932-385f-4e12-a84d-074556811d16 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6808/100945192,v1:172.21.15.138:6809/100945192] [v2:172.21.15.138:6810/100945192,v1:172.21.15.138:6811/100945192] exists,up 79a31103-aabc-450e-8932-cd75be992a2d 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6818/194684740,v1:172.21.15.94:6819/194684740] [v2:172.21.15.94:6820/194684740,v1:172.21.15.94:6821/194684740] exists,up a27a3f85-5246-40dc-bbda-111d7e2f806b 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6816/2535537173,v1:172.21.15.137:6817/2535537173] [v2:172.21.15.137:6818/2535537173,v1:172.21.15.137:6819/2535537173] exists,up 4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6816/3378906429,v1:172.21.15.138:6817/3378906429] [v2:172.21.15.138:6818/3378906429,v1:172.21.15.138:6819/3378906429] exists,up 571dbd3c-a8f3-425f-a34c-f0f2c168907a 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6826/1308519263,v1:172.21.15.94:6827/1308519263] [v2:172.21.15.94:6828/1308519263,v1:172.21.15.94:6829/1308519263] exists,up f0b7d2e9-2dd4-4dbf-befd-e860062b42ae 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.137:6824/3255175900,v1:172.21.15.137:6825/3255175900] [v2:172.21.15.137:6826/3255175900,v1:172.21.15.137:6827/3255175900] exists,up 3a0daed0-84c8-4c9f-977e-d2502f39441e 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6824/883275794,v1:172.21.15.138:6825/883275794] [v2:172.21.15.138:6826/883275794,v1:172.21.15.138:6827/883275794] exists,up 3b938f69-8356-40c2-9a76-45bed8d6a261 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.10 [8,11] 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.7 [3,9] 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.c [3,9] 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.11 [5,9] 2024-04-25T15:01:53.021 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.17 [4,6] 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.21 [4,7] 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.23 [4,8] 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.39 [5,8] 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3440614830 expires 2024-04-26T14:52:01.342497+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/1490569056 expires 2024-04-26T14:52:01.342497+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/1217751898 expires 2024-04-26T14:52:01.342497+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/1217751898 expires 2024-04-26T14:52:01.342497+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/17546499 expires 2024-04-26T14:51:27.240380+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/1137535670 expires 2024-04-26T14:51:27.240380+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2519672579 expires 2024-04-26T14:51:27.240380+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/312144263 expires 2024-04-26T14:51:27.240380+0000 2024-04-25T15:01:53.022 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/17546499 expires 2024-04-26T14:51:27.240380+0000 2024-04-25T15:01:53.023 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3116385060 expires 2024-04-26T14:52:01.342497+0000 2024-04-25T15:01:53.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7ee80a3cc0 msgr2=0x7f7ee80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7ee80a3cc0 0x7f7ee80a6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009680 tx=0x7f7eec009260 comp rx=0 tx=0).stop 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f001090f0 msgr2=0x7f7f0019f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f001090f0 0x7f7f0019f230 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e980 tx=0x7f7ef400ec90 comp rx=0 tx=0).stop 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 shutdown_connections 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f7ee80a3cc0 0x7f7ee80a6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f0010ac90 0x7f7f001a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7f001090f0 0x7f7f0019f230 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 --2- 172.21.15.94:0/2963475149 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7f0010a2f0 0x7f7f0019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.020+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 >> 172.21.15.94:0/2963475149 conn(0x7f7f000780c0 msgr2=0x7f7f0010d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.021+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 shutdown_connections 2024-04-25T15:01:53.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:53.021+0000 7f7f08600700 1 -- 172.21.15.94:0/2963475149 wait complete. 2024-04-25T15:01:53.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:53 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2963475149' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T15:01:53.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:53 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2963475149' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T15:01:53.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:53 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2963475149' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T15:01:53.853 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T15:01:53.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:54.079 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:54 smithi094 ceph-mon[27862]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 -- 172.21.15.94:0/2998147826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0102a10 msgr2=0x7f0ac0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 --2- 172.21.15.94:0/2998147826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0102a10 0x7f0ac0104e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009900 tx=0x7f0ab402a410 comp rx=0 tx=0).stop 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 -- 172.21.15.94:0/2998147826 shutdown_connections 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 --2- 172.21.15.94:0/2998147826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ac0107c70 0x7f0ac010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 --2- 172.21.15.94:0/2998147826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 0x7f0ac0107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 --2- 172.21.15.94:0/2998147826 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0102a10 0x7f0ac0104e00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 -- 172.21.15.94:0/2998147826 >> 172.21.15.94:0/2998147826 conn(0x7f0ac00fc400 msgr2=0x7f0ac00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 -- 172.21.15.94:0/2998147826 shutdown_connections 2024-04-25T15:01:54.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.087+0000 7f0ac8800640 1 -- 172.21.15.94:0/2998147826 wait complete. 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.088+0000 7f0ac8800640 1 Processor -- start 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac8800640 1 -- start start 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac8800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ac0102a10 0x7f0ac019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac8800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 0x7f0ac019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac8800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0107c70 0x7f0ac01a1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac8800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac010db30 con 0x7f0ac0105340 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac8800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac010d9b0 con 0x7f0ac0107c70 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac8800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac010dcb0 con 0x7f0ac0102a10 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 0x7f0ac019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 0x7f0ac019a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58192/0 (socket says 172.21.15.94:58192) 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac5400640 1 -- 172.21.15.94:0/890599257 learned_addr learned my addr 172.21.15.94:0/890599257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac6800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0107c70 0x7f0ac01a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.089+0000 7f0ac5e00640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ac0102a10 0x7f0ac019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac5400640 1 -- 172.21.15.94:0/890599257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ac0102a10 msgr2=0x7f0ac019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac5400640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ac0102a10 0x7f0ac019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac5400640 1 -- 172.21.15.94:0/890599257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0107c70 msgr2=0x7f0ac01a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac5400640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0107c70 0x7f0ac01a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac5400640 1 -- 172.21.15.94:0/890599257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac01a2000 con 0x7f0ac0105340 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac6800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0107c70 0x7f0ac01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac5400640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 0x7f0ac019a920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000d770 tx=0x7f0ab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0abaa00640 1 -- 172.21.15.94:0/890599257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0017070 con 0x7f0ac0105340 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac01a22f0 con 0x7f0ac0105340 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0abaa00640 1 -- 172.21.15.94:0/890599257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab0009c30 con 0x7f0ac0105340 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0abaa00640 1 -- 172.21.15.94:0/890599257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000f620 con 0x7f0ac0105340 2024-04-25T15:01:54.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.090+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac01a2840 con 0x7f0ac0105340 2024-04-25T15:01:54.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.092+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac01097c0 con 0x7f0ac0105340 2024-04-25T15:01:54.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.094+0000 7f0abaa00640 1 -- 172.21.15.94:0/890599257 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0ab001a070 con 0x7f0ac0105340 2024-04-25T15:01:54.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.095+0000 7f0abaa00640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0aa80a3bb0 0x7f0aa80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.095+0000 7f0abaa00640 1 -- 172.21.15.94:0/890599257 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f0ab00cfef0 con 0x7f0ac0105340 2024-04-25T15:01:54.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.095+0000 7f0ac5e00640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0aa80a3bb0 0x7f0aa80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.096+0000 7f0ac5e00640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0aa80a3bb0 0x7f0aa80a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4031500 tx=0x7f0ab4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:54.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.101+0000 7f0abaa00640 1 -- 172.21.15.94:0/890599257 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ab0009de0 con 0x7f0ac0105340 2024-04-25T15:01:54.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.302+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ac0062b50 con 0x7f0ac0105340 2024-04-25T15:01:54.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.304+0000 7f0abaa00640 1 -- 172.21.15.94:0/890599257 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7f0ab009c240 con 0x7f0ac0105340 2024-04-25T15:01:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:54.305 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:54.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.306+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0aa80a3bb0 msgr2=0x7f0aa80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0aa80a3bb0 0x7f0aa80a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4031500 tx=0x7f0ab4005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:54.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 msgr2=0x7f0ac019a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 0x7f0ac019a920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000d770 tx=0x7f0ab000dc30 comp rx=0 tx=0).stop 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 shutdown_connections 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f0aa80a3bb0 0x7f0aa80a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0ac0107c70 0x7f0ac01a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ac0105340 0x7f0ac019a920 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 --2- 172.21.15.94:0/890599257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ac0102a10 0x7f0ac019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 >> 172.21.15.94:0/890599257 conn(0x7f0ac00fc400 msgr2=0x7f0ac01020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 shutdown_connections 2024-04-25T15:01:54.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.307+0000 7f0ac8800640 1 -- 172.21.15.94:0/890599257 wait complete. 2024-04-25T15:01:54.331 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:01:54.331 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:01:54.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:54 smithi138 ceph-mon[32875]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:54.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:54 smithi137 ceph-mon[32439]: pgmap v374: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 -- 172.21.15.94:0/3198578824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98104090 msgr2=0x7f4f981044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/3198578824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98104090 0x7f4f981044b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c009900 tx=0x7f4f8c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 -- 172.21.15.94:0/3198578824 shutdown_connections 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/3198578824 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98105c40 0x7f4f9810c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/3198578824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98105280 0x7f4f98105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/3198578824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98104090 0x7f4f981044b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 -- 172.21.15.94:0/3198578824 >> 172.21.15.94:0/3198578824 conn(0x7f4f980ff600 msgr2=0x7f4f98101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 -- 172.21.15.94:0/3198578824 shutdown_connections 2024-04-25T15:01:54.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.523+0000 7f4f9ea00640 1 -- 172.21.15.94:0/3198578824 wait complete. 2024-04-25T15:01:54.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.524+0000 7f4f9ea00640 1 Processor -- start 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.524+0000 7f4f9ea00640 1 -- start start 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98104090 0x7f4f9819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9ea00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98105280 0x7f4f9819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9ea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98105c40 0x7f4f981a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9ea00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810fc20 con 0x7f4f98105c40 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9ea00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810faa0 con 0x7f4f98105280 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9810fda0 con 0x7f4f98104090 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98105c40 0x7f4f981a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98105c40 0x7f4f981a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58216/0 (socket says 172.21.15.94:58216) 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 -- 172.21.15.94:0/2879987202 learned_addr learned my addr 172.21.15.94:0/2879987202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9d600640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98104090 0x7f4f9819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9cc00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98105280 0x7f4f9819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 -- 172.21.15.94:0/2879987202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98104090 msgr2=0x7f4f9819a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98104090 0x7f4f9819a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 -- 172.21.15.94:0/2879987202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98105280 msgr2=0x7f4f9819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98105280 0x7f4f9819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9e000640 1 -- 172.21.15.94:0/2879987202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f981a1ed0 con 0x7f4f98105c40 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.525+0000 7f4f9cc00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98105280 0x7f4f9819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.526+0000 7f4f9e000640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98105c40 0x7f4f981a1860 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000eb10 tx=0x7f4f9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.526+0000 7f4f96000640 1 -- 172.21.15.94:0/2879987202 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90004b20 con 0x7f4f98105c40 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.526+0000 7f4f96000640 1 -- 172.21.15.94:0/2879987202 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f90004cc0 con 0x7f4f98105c40 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.526+0000 7f4f96000640 1 -- 172.21.15.94:0/2879987202 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90010670 con 0x7f4f98105c40 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.526+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f981a21c0 con 0x7f4f98105c40 2024-04-25T15:01:54.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.526+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f981a26c0 con 0x7f4f98105c40 2024-04-25T15:01:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.527+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9810b450 con 0x7f4f98105c40 2024-04-25T15:01:54.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.529+0000 7f4f96000640 1 -- 172.21.15.94:0/2879987202 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4f90005020 con 0x7f4f98105c40 2024-04-25T15:01:54.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.530+0000 7f4f96000640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4f800a3bb0 0x7f4f800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.531+0000 7f4f96000640 1 -- 172.21.15.94:0/2879987202 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f4f900d09e0 con 0x7f4f98105c40 2024-04-25T15:01:54.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.531+0000 7f4f9d600640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4f800a3bb0 0x7f4f800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.531+0000 7f4f9d600640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4f800a3bb0 0x7f4f800a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005ab0 tx=0x7f4f8c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:54.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.535+0000 7f4f96000640 1 -- 172.21.15.94:0/2879987202 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f9009d380 con 0x7f4f98105c40 2024-04-25T15:01:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.738+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f98062b50 con 0x7f4f98105c40 2024-04-25T15:01:54.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.739+0000 7f4f96000640 1 -- 172.21.15.94:0/2879987202 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f4f9009cc80 con 0x7f4f98105c40 2024-04-25T15:01:54.740 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:01:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4f800a3bb0 msgr2=0x7f4f800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4f800a3bb0 0x7f4f800a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005ab0 tx=0x7f4f8c005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98105c40 msgr2=0x7f4f981a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98105c40 0x7f4f981a1860 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000eb10 tx=0x7f4f9000c2d0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 shutdown_connections 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f4f800a3bb0 0x7f4f800a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f98105c40 0x7f4f981a1860 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4f98105280 0x7f4f9819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 --2- 172.21.15.94:0/2879987202 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f98104090 0x7f4f9819a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 >> 172.21.15.94:0/2879987202 conn(0x7f4f980ff600 msgr2=0x7f4f98108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 shutdown_connections 2024-04-25T15:01:54.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.742+0000 7f4f9ea00640 1 -- 172.21.15.94:0/2879987202 wait complete. 2024-04-25T15:01:54.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:01:54.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.966+0000 7fbae5200640 1 -- 172.21.15.94:0/3143419095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0108490 msgr2=0x7fbae01088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.966+0000 7fbae5200640 1 --2- 172.21.15.94:0/3143419095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0108490 0x7fbae01088b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009900 tx=0x7fbad002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.966+0000 7fbae5200640 1 -- 172.21.15.94:0/3143419095 shutdown_connections 2024-04-25T15:01:54.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.966+0000 7fbae5200640 1 --2- 172.21.15.94:0/3143419095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae010a040 0x7fbae01108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.966+0000 7fbae5200640 1 --2- 172.21.15.94:0/3143419095 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae0109680 0x7fbae0109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.966+0000 7fbae5200640 1 --2- 172.21.15.94:0/3143419095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0108490 0x7fbae01088b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.966+0000 7fbae5200640 1 -- 172.21.15.94:0/3143419095 >> 172.21.15.94:0/3143419095 conn(0x7fbae0077420 msgr2=0x7fbae0079860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.967+0000 7fbae5200640 1 -- 172.21.15.94:0/3143419095 shutdown_connections 2024-04-25T15:01:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.967+0000 7fbae5200640 1 -- 172.21.15.94:0/3143419095 wait complete. 2024-04-25T15:01:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 Processor -- start 2024-04-25T15:01:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 -- start start 2024-04-25T15:01:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae0108490 0x7fbae019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0109680 0x7fbae019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae010a040 0x7fbae01a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0114020 con 0x7fbae0109680 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0113ea0 con 0x7fbae010a040 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbae5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01141a0 con 0x7fbae0108490 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.968+0000 7fbadf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae0108490 0x7fbae019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbad7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0109680 0x7fbae019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae0108490 0x7fbae019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:48404/0 (socket says 172.21.15.94:48404) 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 -- 172.21.15.94:0/3827368862 learned_addr learned my addr 172.21.15.94:0/3827368862 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadfe00640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae010a040 0x7fbae01a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 -- 172.21.15.94:0/3827368862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae010a040 msgr2=0x7fbae01a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae010a040 0x7fbae01a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 -- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0109680 msgr2=0x7fbae019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:54.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0109680 0x7fbae019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:54.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 -- 172.21.15.94:0/3827368862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae01a62c0 con 0x7fbae0108490 2024-04-25T15:01:54.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadf400640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae0108490 0x7fbae019e6a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbad0031500 tx=0x7fbad0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:54.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.969+0000 7fbadcc00640 1 -- 172.21.15.94:0/3827368862 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad003f070 con 0x7fbae0108490 2024-04-25T15:01:54.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.970+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae01a6550 con 0x7fbae0108490 2024-04-25T15:01:54.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.970+0000 7fbadcc00640 1 -- 172.21.15.94:0/3827368862 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad0031c50 con 0x7fbae0108490 2024-04-25T15:01:54.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.970+0000 7fbadcc00640 1 -- 172.21.15.94:0/3827368862 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0035030 con 0x7fbae0108490 2024-04-25T15:01:54.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.970+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae01a6a40 con 0x7fbae0108490 2024-04-25T15:01:54.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.971+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae010f830 con 0x7fbae0108490 2024-04-25T15:01:54.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.973+0000 7fbadcc00640 1 -- 172.21.15.94:0/3827368862 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbad0042070 con 0x7fbae0108490 2024-04-25T15:01:54.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.974+0000 7fbadcc00640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbac40a3bb0 0x7fbac40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:54.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.974+0000 7fbadcc00640 1 -- 172.21.15.94:0/3827368862 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fbad00f1270 con 0x7fbae0108490 2024-04-25T15:01:54.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.974+0000 7fbad7e00640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbac40a3bb0 0x7fbac40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:54.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.975+0000 7fbad7e00640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbac40a3bb0 0x7fbac40a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbac80095f0 tx=0x7fbac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:54.977+0000 7fbadcc00640 1 -- 172.21.15.94:0/3827368862 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbad00bdc10 con 0x7fbae0108490 2024-04-25T15:01:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.175+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbae0062b50 con 0x7fbae0108490 2024-04-25T15:01:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.176+0000 7fbadcc00640 1 -- 172.21.15.94:0/3827368862 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fbad00bd510 con 0x7fbae0108490 2024-04-25T15:01:55.177 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:01:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbac40a3bb0 msgr2=0x7fbac40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbac40a3bb0 0x7fbac40a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbac80095f0 tx=0x7fbac8005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae0108490 msgr2=0x7fbae019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae0108490 0x7fbae019e6a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbad0031500 tx=0x7fbad0003c10 comp rx=0 tx=0).stop 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 shutdown_connections 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fbac40a3bb0 0x7fbac40a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fbae010a040 0x7fbae01a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbae0109680 0x7fbae019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 --2- 172.21.15.94:0/3827368862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae0108490 0x7fbae019e6a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 >> 172.21.15.94:0/3827368862 conn(0x7fbae0077420 msgr2=0x7fbae010ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 shutdown_connections 2024-04-25T15:01:55.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.179+0000 7fbae5200640 1 -- 172.21.15.94:0/3827368862 wait complete. 2024-04-25T15:01:55.198 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:01:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.404+0000 7fb9d1a00640 1 -- 172.21.15.94:0/3649364925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc101e60 msgr2=0x7fb9cc102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.404+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/3649364925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc101e60 0x7fb9cc102280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009900 tx=0x7fb9bc02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.404+0000 7fb9d1a00640 1 -- 172.21.15.94:0/3649364925 shutdown_connections 2024-04-25T15:01:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.404+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/3649364925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9cc103970 0x7fb9cc10a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.404+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/3649364925 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9cc102fb0 0x7fb9cc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.404+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/3649364925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc101e60 0x7fb9cc102280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.404+0000 7fb9d1a00640 1 -- 172.21.15.94:0/3649364925 >> 172.21.15.94:0/3649364925 conn(0x7fb9cc0fd3f0 msgr2=0x7fb9cc0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:55.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.405+0000 7fb9d1a00640 1 -- 172.21.15.94:0/3649364925 shutdown_connections 2024-04-25T15:01:55.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.405+0000 7fb9d1a00640 1 -- 172.21.15.94:0/3649364925 wait complete. 2024-04-25T15:01:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 Processor -- start 2024-04-25T15:01:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 -- start start 2024-04-25T15:01:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9cc101e60 0x7fb9cc075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9cc102fb0 0x7fb9cc074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc103970 0x7fb9cc074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10da00 con 0x7fb9cc101e60 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10d880 con 0x7fb9cc102fb0 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.406+0000 7fb9d1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10db80 con 0x7fb9cc103970 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc103970 0x7fb9cc074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9cb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9cc102fb0 0x7fb9cc074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9cbe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9cc101e60 0x7fb9cc075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc103970 0x7fb9cc074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:48418/0 (socket says 172.21.15.94:48418) 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9cb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9cc102fb0 0x7fb9cc074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54420/0 (socket says 172.21.15.94:54420) 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 -- 172.21.15.94:0/439191978 learned_addr learned my addr 172.21.15.94:0/439191978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 -- 172.21.15.94:0/439191978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9cc102fb0 msgr2=0x7fb9cc074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9cc102fb0 0x7fb9cc074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 -- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9cc101e60 msgr2=0x7fb9cc075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9cc101e60 0x7fb9cc075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 -- 172.21.15.94:0/439191978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc074dc0 con 0x7fb9cc103970 2024-04-25T15:01:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.407+0000 7fb9d1000640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc103970 0x7fb9cc074880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400efd0 tx=0x7fb9c400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.408+0000 7fb9c8c00640 1 -- 172.21.15.94:0/439191978 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4003b70 con 0x7fb9cc103970 2024-04-25T15:01:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.408+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc0750b0 con 0x7fb9cc103970 2024-04-25T15:01:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.408+0000 7fb9c8c00640 1 -- 172.21.15.94:0/439191978 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c4003d10 con 0x7fb9cc103970 2024-04-25T15:01:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.408+0000 7fb9c8c00640 1 -- 172.21.15.94:0/439191978 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c40108a0 con 0x7fb9cc103970 2024-04-25T15:01:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.408+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1a6b30 con 0x7fb9cc103970 2024-04-25T15:01:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.409+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9cc109170 con 0x7fb9cc103970 2024-04-25T15:01:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.411+0000 7fb9c8c00640 1 -- 172.21.15.94:0/439191978 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb9c4004020 con 0x7fb9cc103970 2024-04-25T15:01:55.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.412+0000 7fb9c8c00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb9b00a3bb0 0x7fb9b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.412+0000 7fb9c8c00640 1 -- 172.21.15.94:0/439191978 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb9c40d0820 con 0x7fb9cc103970 2024-04-25T15:01:55.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.413+0000 7fb9cbe00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb9b00a3bb0 0x7fb9b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.413+0000 7fb9cbe00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb9b00a3bb0 0x7fb9b00a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc031500 tx=0x7fb9bc005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:55.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.417+0000 7fb9c8c00640 1 -- 172.21.15.94:0/439191978 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb9c409d1c0 con 0x7fb9cc103970 2024-04-25T15:01:55.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/890599257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:55.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2879987202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:55.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:55 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/890599257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:55.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:55 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2879987202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:55.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:55 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/890599257' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:55.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:55 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2879987202' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:55.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.620+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9cc062b50 con 0x7fb9cc103970 2024-04-25T15:01:55.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.621+0000 7fb9c8c00640 1 -- 172.21.15.94:0/439191978 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fb9c409cac0 con 0x7fb9cc103970 2024-04-25T15:01:55.621 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:01:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb9b00a3bb0 msgr2=0x7fb9b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb9b00a3bb0 0x7fb9b00a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc031500 tx=0x7fb9bc005a20 comp rx=0 tx=0).stop 2024-04-25T15:01:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc103970 msgr2=0x7fb9cc074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc103970 0x7fb9cc074880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400efd0 tx=0x7fb9c400c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 shutdown_connections 2024-04-25T15:01:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb9b00a3bb0 0x7fb9b00a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9cc103970 0x7fb9cc074880 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb9cc102fb0 0x7fb9cc074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 --2- 172.21.15.94:0/439191978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9cc101e60 0x7fb9cc075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.623+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 >> 172.21.15.94:0/439191978 conn(0x7fb9cc0fd3f0 msgr2=0x7fb9cc106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.624+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 shutdown_connections 2024-04-25T15:01:55.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.624+0000 7fb9d1a00640 1 -- 172.21.15.94:0/439191978 wait complete. 2024-04-25T15:01:55.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:55.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.839+0000 7f884f200640 1 -- 172.21.15.94:0/2325048367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88501083d0 msgr2=0x7f88501087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.839+0000 7f884f200640 1 --2- 172.21.15.94:0/2325048367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88501083d0 0x7f88501087f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8840009900 tx=0x7f884002a410 comp rx=0 tx=0).stop 2024-04-25T15:01:55.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.839+0000 7f884f200640 1 -- 172.21.15.94:0/2325048367 shutdown_connections 2024-04-25T15:01:55.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.840+0000 7f884f200640 1 --2- 172.21.15.94:0/2325048367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8850109f80 0x7f8850110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.840+0000 7f884f200640 1 --2- 172.21.15.94:0/2325048367 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501095c0 0x7f8850109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.840+0000 7f884f200640 1 --2- 172.21.15.94:0/2325048367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88501083d0 0x7f88501087f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.840+0000 7f884f200640 1 -- 172.21.15.94:0/2325048367 >> 172.21.15.94:0/2325048367 conn(0x7f8850077390 msgr2=0x7f88500797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:55.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.840+0000 7f884f200640 1 -- 172.21.15.94:0/2325048367 shutdown_connections 2024-04-25T15:01:55.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.840+0000 7f884f200640 1 -- 172.21.15.94:0/2325048367 wait complete. 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 Processor -- start 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 -- start start 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501083d0 0x7f885019e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88501095c0 0x7f885019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8850109f80 0x7f88501a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113f60 con 0x7f88501095c0 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f8850113de0 con 0x7f88501083d0 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.841+0000 7f884f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88501140e0 con 0x7f8850109f80 2024-04-25T15:01:55.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f8845400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88501095c0 0x7f885019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8850109f80 0x7f88501a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501083d0 0x7f885019e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501083d0 0x7f885019e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:54430/0 (socket says 172.21.15.94:54430) 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 -- 172.21.15.94:0/2666962196 learned_addr learned my addr 172.21.15.94:0/2666962196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8850109f80 0x7f88501a5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:48448/0 (socket says 172.21.15.94:48448) 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 -- 172.21.15.94:0/2666962196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8850109f80 msgr2=0x7f88501a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8850109f80 0x7f88501a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 -- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88501095c0 msgr2=0x7f885019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88501095c0 0x7f885019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 -- 172.21.15.94:0/2666962196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88501a6230 con 0x7f88501083d0 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f884de00640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501083d0 0x7f885019e610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8840031500 tx=0x7f8840003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.842+0000 7f8847400640 1 -- 172.21.15.94:0/2666962196 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f884003f070 con 0x7f88501083d0 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.843+0000 7f8847400640 1 -- 172.21.15.94:0/2666962196 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8840031c50 con 0x7f88501083d0 2024-04-25T15:01:55.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.843+0000 7f8847400640 1 -- 172.21.15.94:0/2666962196 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840035030 con 0x7f88501083d0 2024-04-25T15:01:55.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.843+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88501a64c0 con 0x7f88501083d0 2024-04-25T15:01:55.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.843+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88501a6960 con 0x7f88501083d0 2024-04-25T15:01:55.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.844+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885010f740 con 0x7f88501083d0 2024-04-25T15:01:55.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.846+0000 7f8847400640 1 -- 172.21.15.94:0/2666962196 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8840042070 con 0x7f88501083d0 2024-04-25T15:01:55.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.847+0000 7f8847400640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88340a3bb0 0x7f88340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:55.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.847+0000 7f8847400640 1 -- 172.21.15.94:0/2666962196 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f88400f1230 con 0x7f88501083d0 2024-04-25T15:01:55.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.847+0000 7f8845400640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88340a3bb0 0x7f88340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:55.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.848+0000 7f8845400640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88340a3bb0 0x7f88340a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f885019fb20 tx=0x7f8838005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:55.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:55.849+0000 7f8847400640 1 -- 172.21.15.94:0/2666962196 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88400bd580 con 0x7f88501083d0 2024-04-25T15:01:56.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.096+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f885002d5c0 con 0x7f88501083d0 2024-04-25T15:01:56.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.098+0000 7f8847400640 1 -- 172.21.15.94:0/2666962196 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f88400c1880 con 0x7f88501083d0 2024-04-25T15:01:56.098 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:56.098 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:01:56.099 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.101+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88340a3bb0 msgr2=0x7f88340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.101+0000 7f884f200640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88340a3bb0 0x7f88340a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f885019fb20 tx=0x7f8838005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.101+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501083d0 msgr2=0x7f885019e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.101+0000 7f884f200640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501083d0 0x7f885019e610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8840031500 tx=0x7f8840003c10 comp rx=0 tx=0).stop 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 shutdown_connections 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f88340a3bb0 0x7f88340a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8850109f80 0x7f88501a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88501095c0 0x7f885019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 --2- 172.21.15.94:0/2666962196 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88501083d0 0x7f885019e610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 >> 172.21.15.94:0/2666962196 conn(0x7f8850077390 msgr2=0x7f885010c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 shutdown_connections 2024-04-25T15:01:56.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.102+0000 7f884f200640 1 -- 172.21.15.94:0/2666962196 wait complete. 2024-04-25T15:01:56.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 -- 172.21.15.94:0/3144538236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0102850 msgr2=0x7fb0b0104c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 --2- 172.21.15.94:0/3144538236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0102850 0x7fb0b0104c40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0009860 tx=0x7fb0a002a340 comp rx=0 tx=0).stop 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 -- 172.21.15.94:0/3144538236 shutdown_connections 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 --2- 172.21.15.94:0/3144538236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0b0107ab0 0x7fb0b0109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 --2- 172.21.15.94:0/3144538236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0105180 0x7fb0b0107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 --2- 172.21.15.94:0/3144538236 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0102850 0x7fb0b0104c40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 -- 172.21.15.94:0/3144538236 >> 172.21.15.94:0/3144538236 conn(0x7fb0b00fc220 msgr2=0x7fb0b00fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 -- 172.21.15.94:0/3144538236 shutdown_connections 2024-04-25T15:01:56.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.312+0000 7fb0af200640 1 -- 172.21.15.94:0/3144538236 wait complete. 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.313+0000 7fb0af200640 1 Processor -- start 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.313+0000 7fb0af200640 1 -- start start 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0af200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0102850 0x7fb0b01021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0af200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0105180 0x7fb0b0100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0af200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0b0107ab0 0x7fb0b0100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0af200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010d920 con 0x7fb0b0102850 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0af200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010d7a0 con 0x7fb0b0105180 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0af200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010daa0 con 0x7fb0b0107ab0 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0102850 0x7fb0b01021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0102850 0x7fb0b01021c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58296/0 (socket says 172.21.15.94:58296) 2024-04-25T15:01:56.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 -- 172.21.15.94:0/3651936542 learned_addr learned my addr 172.21.15.94:0/3651936542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ae800640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0b0107ab0 0x7fb0b0100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0a5400640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0105180 0x7fb0b0100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 -- 172.21.15.94:0/3651936542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0b0107ab0 msgr2=0x7fb0b0100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0b0107ab0 0x7fb0b0100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 -- 172.21.15.94:0/3651936542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0105180 msgr2=0x7fb0b0100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0105180 0x7fb0b0100800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.314+0000 7fb0ade00640 1 -- 172.21.15.94:0/3651936542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b01015d0 con 0x7fb0b0102850 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.315+0000 7fb0ade00640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0102850 0x7fb0b01021c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00057c0 tx=0x7fb0a0032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.315+0000 7fb0a7400640 1 -- 172.21.15.94:0/3651936542 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0037040 con 0x7fb0b0102850 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.315+0000 7fb0a7400640 1 -- 172.21.15.94:0/3651936542 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a0030bd0 con 0x7fb0b0102850 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.315+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b01a20e0 con 0x7fb0b0102850 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.315+0000 7fb0a7400640 1 -- 172.21.15.94:0/3651936542 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0005280 con 0x7fb0b0102850 2024-04-25T15:01:56.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.315+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01a25d0 con 0x7fb0b0102850 2024-04-25T15:01:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.316+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b00fdd10 con 0x7fb0b0102850 2024-04-25T15:01:56.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.318+0000 7fb0a7400640 1 -- 172.21.15.94:0/3651936542 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb0a0043020 con 0x7fb0b0102850 2024-04-25T15:01:56.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.319+0000 7fb0a7400640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb0940a3bb0 0x7fb0940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.320+0000 7fb0a7400640 1 -- 172.21.15.94:0/3651936542 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fb0a00efa50 con 0x7fb0b0102850 2024-04-25T15:01:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.320+0000 7fb0a5400640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb0940a3bb0 0x7fb0940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.320+0000 7fb0a5400640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb0940a3bb0 0x7fb0940a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0101f30 tx=0x7fb098009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.324+0000 7fb0a7400640 1 -- 172.21.15.94:0/3651936542 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0a00bc3f0 con 0x7fb0b0102850 2024-04-25T15:01:56.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:56 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3827368862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:56.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:56 smithi138 ceph-mon[32875]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:56.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:56 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/439191978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:56.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:56 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3827368862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:56.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:56 smithi094 ceph-mon[27862]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:56.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:56 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/439191978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:56.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:56 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3827368862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:56.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:56 smithi137 ceph-mon[32439]: pgmap v375: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:01:56.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:56 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/439191978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:56.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.526+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0b0062b50 con 0x7fb0b0102850 2024-04-25T15:01:56.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.528+0000 7fb0a7400640 1 -- 172.21.15.94:0/3651936542 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7fb0a00328c0 con 0x7fb0b0102850 2024-04-25T15:01:56.529 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:56.529 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb0940a3bb0 msgr2=0x7fb0940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb0940a3bb0 0x7fb0940a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0101f30 tx=0x7fb098009260 comp rx=0 tx=0).stop 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0102850 msgr2=0x7fb0b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0102850 0x7fb0b01021c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00057c0 tx=0x7fb0a0032da0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 shutdown_connections 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fb0940a3bb0 0x7fb0940a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0b0107ab0 0x7fb0b0100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb0b0105180 0x7fb0b0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 --2- 172.21.15.94:0/3651936542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0b0102850 0x7fb0b01021c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 >> 172.21.15.94:0/3651936542 conn(0x7fb0b00fc220 msgr2=0x7fb0b01032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 shutdown_connections 2024-04-25T15:01:56.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.531+0000 7fb0af200640 1 -- 172.21.15.94:0/3651936542 wait complete. 2024-04-25T15:01:56.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:01:56.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 -- 172.21.15.94:0/3347899461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834105c40 msgr2=0x7fd83410c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 --2- 172.21.15.94:0/3347899461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834105c40 0x7fd83410c4d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009900 tx=0x7fd82c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 -- 172.21.15.94:0/3347899461 shutdown_connections 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 --2- 172.21.15.94:0/3347899461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834105c40 0x7fd83410c4d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 --2- 172.21.15.94:0/3347899461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd834105280 0x7fd834105700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 --2- 172.21.15.94:0/3347899461 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834104090 0x7fd8341044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 -- 172.21.15.94:0/3347899461 >> 172.21.15.94:0/3347899461 conn(0x7fd8340ff600 msgr2=0x7fd834101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 -- 172.21.15.94:0/3347899461 shutdown_connections 2024-04-25T15:01:56.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.745+0000 7fd83a200640 1 -- 172.21.15.94:0/3347899461 wait complete. 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.746+0000 7fd83a200640 1 Processor -- start 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.746+0000 7fd83a200640 1 -- start start 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd83a200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104090 0x7fd83419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd83a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd834105280 0x7fd83419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd83a200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834105c40 0x7fd8341a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd83a200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410fc20 con 0x7fd834104090 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd83a200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410faa0 con 0x7fd834105c40 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd83a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd83410fda0 con 0x7fd834105280 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104090 0x7fd83419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104090 0x7fd83419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58316/0 (socket says 172.21.15.94:58316) 2024-04-25T15:01:56.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 -- 172.21.15.94:0/1323377834 learned_addr learned my addr 172.21.15.94:0/1323377834 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd839800640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834105c40 0x7fd8341a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd833e00640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd834105280 0x7fd83419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 -- 172.21.15.94:0/1323377834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd834105280 msgr2=0x7fd83419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd834105280 0x7fd83419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 -- 172.21.15.94:0/1323377834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834105c40 msgr2=0x7fd8341a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834105c40 0x7fd8341a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 -- 172.21.15.94:0/1323377834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8341a1e10 con 0x7fd834104090 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.747+0000 7fd838e00640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104090 0x7fd83419a1f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ebc0 tx=0x7fd82800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.748+0000 7fd831600640 1 -- 172.21.15.94:0/1323377834 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828003b70 con 0x7fd834104090 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.748+0000 7fd831600640 1 -- 172.21.15.94:0/1323377834 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd828003d10 con 0x7fd834104090 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.748+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8341a2100 con 0x7fd834104090 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.748+0000 7fd831600640 1 -- 172.21.15.94:0/1323377834 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd828019800 con 0x7fd834104090 2024-04-25T15:01:56.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.748+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341a26a0 con 0x7fd834104090 2024-04-25T15:01:56.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.749+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd83410b3b0 con 0x7fd834104090 2024-04-25T15:01:56.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.751+0000 7fd831600640 1 -- 172.21.15.94:0/1323377834 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd8280199a0 con 0x7fd834104090 2024-04-25T15:01:56.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.752+0000 7fd831600640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd81c0a3c30 0x7fd81c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:56.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.753+0000 7fd831600640 1 -- 172.21.15.94:0/1323377834 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd8280d0c00 con 0x7fd834104090 2024-04-25T15:01:56.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.753+0000 7fd833e00640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd81c0a3c30 0x7fd81c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:56.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.753+0000 7fd833e00640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd81c0a3c30 0x7fd81c0a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd83419b700 tx=0x7fd820005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:56.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.757+0000 7fd831600640 1 -- 172.21.15.94:0/1323377834 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd82809d5e0 con 0x7fd834104090 2024-04-25T15:01:56.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.945+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd834062b50 con 0x7fd834104090 2024-04-25T15:01:56.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.946+0000 7fd831600640 1 -- 172.21.15.94:0/1323377834 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd82809cee0 con 0x7fd834104090 2024-04-25T15:01:56.947 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:01:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.948+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd81c0a3c30 msgr2=0x7fd81c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.948+0000 7fd83a200640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd81c0a3c30 0x7fd81c0a60f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd83419b700 tx=0x7fd820005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:56.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.948+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104090 msgr2=0x7fd83419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.948+0000 7fd83a200640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104090 0x7fd83419a1f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd82800ebc0 tx=0x7fd82800c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 shutdown_connections 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd81c0a3c30 0x7fd81c0a60f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd834105c40 0x7fd8341a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd834105280 0x7fd83419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 --2- 172.21.15.94:0/1323377834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd834104090 0x7fd83419a1f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 >> 172.21.15.94:0/1323377834 conn(0x7fd8340ff600 msgr2=0x7fd834108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 shutdown_connections 2024-04-25T15:01:56.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:56.949+0000 7fd83a200640 1 -- 172.21.15.94:0/1323377834 wait complete. 2024-04-25T15:01:56.967 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:01:57.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.162+0000 7f93f5400640 1 -- 172.21.15.94:0/309280971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 msgr2=0x7f93f0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.162+0000 7f93f5400640 1 --2- 172.21.15.94:0/309280971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f0104cd0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009900 tx=0x7f93e002a410 comp rx=0 tx=0).stop 2024-04-25T15:01:57.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.162+0000 7f93f5400640 1 -- 172.21.15.94:0/309280971 shutdown_connections 2024-04-25T15:01:57.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.162+0000 7f93f5400640 1 --2- 172.21.15.94:0/309280971 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93f0107b40 0x7f93f0109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.162+0000 7f93f5400640 1 --2- 172.21.15.94:0/309280971 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93f0105210 0x7f93f0107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.162+0000 7f93f5400640 1 --2- 172.21.15.94:0/309280971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f0104cd0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.162+0000 7f93f5400640 1 -- 172.21.15.94:0/309280971 >> 172.21.15.94:0/309280971 conn(0x7f93f00fc2f0 msgr2=0x7f93f00fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.163+0000 7f93f5400640 1 -- 172.21.15.94:0/309280971 shutdown_connections 2024-04-25T15:01:57.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.163+0000 7f93f5400640 1 -- 172.21.15.94:0/309280971 wait complete. 2024-04-25T15:01:57.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 Processor -- start 2024-04-25T15:01:57.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 -- start start 2024-04-25T15:01:57.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93f0105210 0x7f93f01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93f0107b40 0x7f93f019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f93f010da10 con 0x7f93f01028e0 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f93f010d890 con 0x7f93f0107b40 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93f010db90 con 0x7f93f0105210 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93efe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f01980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51714/0 (socket says 172.21.15.94:51714) 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93efe00640 1 -- 172.21.15.94:0/4167835909 learned_addr learned my addr 172.21.15.94:0/4167835909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93f4a00640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93f0107b40 0x7f93f019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.164+0000 7f93ef400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93f0105210 0x7f93f01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93efe00640 1 -- 172.21.15.94:0/4167835909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93f0105210 msgr2=0x7f93f01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93efe00640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93f0105210 0x7f93f01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93efe00640 1 -- 172.21.15.94:0/4167835909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93f0107b40 msgr2=0x7f93f019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93efe00640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93f0107b40 0x7f93f019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93efe00640 1 -- 172.21.15.94:0/4167835909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f019fcd0 con 0x7f93f01028e0 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93efe00640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f01980b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005ab0 tx=0x7f93e0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93ecc00640 1 -- 172.21.15.94:0/4167835909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e003f070 con 0x7f93f01028e0 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93ecc00640 1 -- 172.21.15.94:0/4167835909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e0031c50 con 0x7f93f01028e0 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f019ff60 con 0x7f93f01028e0 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.165+0000 7f93ecc00640 1 -- 172.21.15.94:0/4167835909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0035030 con 0x7f93f01028e0 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.166+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f01a0370 con 0x7f93f01028e0 2024-04-25T15:01:57.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.167+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f00fdde0 con 0x7f93f01028e0 2024-04-25T15:01:57.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.169+0000 7f93ecc00640 1 -- 172.21.15.94:0/4167835909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f93e0042070 con 0x7f93f01028e0 2024-04-25T15:01:57.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.170+0000 7f93ecc00640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93d40a3bb0 0x7f93d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.170+0000 7f93ecc00640 1 -- 172.21.15.94:0/4167835909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f93e00f0b00 con 0x7f93f01028e0 2024-04-25T15:01:57.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.170+0000 7f93ef400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93d40a3bb0 0x7f93d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.171+0000 7f93ef400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93d40a3bb0 0x7f93d40a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93f01995c0 tx=0x7f93d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.174+0000 7f93ecc00640 1 -- 172.21.15.94:0/4167835909 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93e00bce90 con 0x7f93f01028e0 2024-04-25T15:01:57.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.361+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f93f0062bf0 con 0x7f93f01028e0 2024-04-25T15:01:57.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.362+0000 7f93ecc00640 1 -- 172.21.15.94:0/4167835909 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f93e00c1190 con 0x7f93f01028e0 2024-04-25T15:01:57.363 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:01:57.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93d40a3bb0 msgr2=0x7f93d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93d40a3bb0 0x7f93d40a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93f01995c0 tx=0x7f93d8005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:57.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 msgr2=0x7f93f01980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f01980b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005ab0 tx=0x7f93e0003b90 comp rx=0 tx=0).stop 2024-04-25T15:01:57.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 shutdown_connections 2024-04-25T15:01:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f93d40a3bb0 0x7f93d40a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f93f0107b40 0x7f93f019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93f0105210 0x7f93f01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 --2- 172.21.15.94:0/4167835909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93f01028e0 0x7f93f01980b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 >> 172.21.15.94:0/4167835909 conn(0x7f93f00fc2f0 msgr2=0x7f93f01031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 shutdown_connections 2024-04-25T15:01:57.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.365+0000 7f93f5400640 1 -- 172.21.15.94:0/4167835909 wait complete. 2024-04-25T15:01:57.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:57 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2666962196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:57 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3651936542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:57 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1323377834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:57.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:01:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:57 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2666962196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:57 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3651936542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:57 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1323377834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:57 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2666962196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:57 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3651936542' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:57 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1323377834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:01:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 -- 172.21.15.94:0/3237759592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd47410a160 msgr2=0x7fd4741109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 --2- 172.21.15.94:0/3237759592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd47410a160 0x7fd4741109f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009be0 tx=0x7fd46c02a6c0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 -- 172.21.15.94:0/3237759592 shutdown_connections 2024-04-25T15:01:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 --2- 172.21.15.94:0/3237759592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd47410a160 0x7fd4741109f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 --2- 172.21.15.94:0/3237759592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 0x7fd474109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 --2- 172.21.15.94:0/3237759592 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd4741085b0 0x7fd4741089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 -- 172.21.15.94:0/3237759592 >> 172.21.15.94:0/3237759592 conn(0x7fd474077550 msgr2=0x7fd474079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.580+0000 7fd47b800640 1 -- 172.21.15.94:0/3237759592 shutdown_connections 2024-04-25T15:01:57.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.581+0000 7fd47b800640 1 -- 172.21.15.94:0/3237759592 wait complete. 2024-04-25T15:01:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.581+0000 7fd47b800640 1 Processor -- start 2024-04-25T15:01:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd47b800640 1 -- start start 2024-04-25T15:01:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd47b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4741085b0 0x7fd47419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd47b800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 0x7fd47419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd47b800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd47410a160 0x7fd4741a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd47b800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd4741140c0 con 0x7fd4741097a0 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd47b800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd474113f40 con 0x7fd47410a160 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd47b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd474114240 con 0x7fd4741085b0 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd46be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 0x7fd47419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd46be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 0x7fd47419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51740/0 (socket says 172.21.15.94:51740) 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd46be00640 1 -- 172.21.15.94:0/1317554613 learned_addr learned my addr 172.21.15.94:0/1317554613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd478e00640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4741085b0 0x7fd47419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.582+0000 7fd479800640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd47410a160 0x7fd4741a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd46be00640 1 -- 172.21.15.94:0/1317554613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4741085b0 msgr2=0x7fd47419e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd46be00640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4741085b0 0x7fd47419e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd46be00640 1 -- 172.21.15.94:0/1317554613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd47410a160 msgr2=0x7fd4741a5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd46be00640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd47410a160 0x7fd4741a5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd46be00640 1 -- 172.21.15.94:0/1317554613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4741a63b0 con 0x7fd4741097a0 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd46be00640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 0x7fd47419ecd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00e9f0 tx=0x7fd45c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd472000640 1 -- 172.21.15.94:0/1317554613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00cd80 con 0x7fd4741097a0 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4741a66a0 con 0x7fd4741097a0 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd472000640 1 -- 172.21.15.94:0/1317554613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45c0108e0 con 0x7fd4741097a0 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.583+0000 7fd472000640 1 -- 172.21.15.94:0/1317554613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c018980 con 0x7fd4741097a0 2024-04-25T15:01:57.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.584+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4741a6bf0 con 0x7fd4741097a0 2024-04-25T15:01:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.585+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd47410f920 con 0x7fd4741097a0 2024-04-25T15:01:57.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.586+0000 7fd472000640 1 -- 172.21.15.94:0/1317554613 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd45c010430 con 0x7fd4741097a0 2024-04-25T15:01:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.588+0000 7fd472000640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4540a3bb0 0x7fd4540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.588+0000 7fd472000640 1 -- 172.21.15.94:0/1317554613 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7fd45c0d0710 con 0x7fd4741097a0 2024-04-25T15:01:57.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.588+0000 7fd478e00640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4540a3bb0 0x7fd4540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:57.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.589+0000 7fd478e00640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4540a3bb0 0x7fd4540a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd464009530 tx=0x7fd464009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:57.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.592+0000 7fd472000640 1 -- 172.21.15.94:0/1317554613 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd45c09ca60 con 0x7fd4741097a0 2024-04-25T15:01:57.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.791+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd474062b50 con 0x7fd4741097a0 2024-04-25T15:01:57.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.792+0000 7fd472000640 1 -- 172.21.15.94:0/1317554613 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7fd45c0a0d60 con 0x7fd4741097a0 2024-04-25T15:01:57.792 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:01:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4540a3bb0 msgr2=0x7fd4540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4540a3bb0 0x7fd4540a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd464009530 tx=0x7fd464009260 comp rx=0 tx=0).stop 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 msgr2=0x7fd47419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 0x7fd47419ecd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00e9f0 tx=0x7fd45c00eeb0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 shutdown_connections 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7fd4540a3bb0 0x7fd4540a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd47410a160 0x7fd4741a5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4741097a0 0x7fd47419ecd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 --2- 172.21.15.94:0/1317554613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4741085b0 0x7fd47419e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 >> 172.21.15.94:0/1317554613 conn(0x7fd474077550 msgr2=0x7fd47410caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 shutdown_connections 2024-04-25T15:01:57.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:57.795+0000 7fd47b800640 1 -- 172.21.15.94:0/1317554613 wait complete. 2024-04-25T15:01:57.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:01:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.008+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3306789747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68076910 msgr2=0x7f5d68076d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.008+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3306789747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68076910 0x7f5d68076d30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5d58009900 tx=0x7f5d5802a3f0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.008+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3306789747 shutdown_connections 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.008+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3306789747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d680784f0 0x7f5d68078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.008+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3306789747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d68077b50 0x7f5d68077fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.008+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3306789747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68076910 0x7f5d68076d30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.008+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3306789747 >> 172.21.15.94:0/3306789747 conn(0x7f5d680ff560 msgr2=0x7f5d68101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.009+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3306789747 shutdown_connections 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.009+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3306789747 wait complete. 2024-04-25T15:01:58.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.009+0000 7f5d6cc00640 1 Processor -- start 2024-04-25T15:01:58.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.009+0000 7f5d6cc00640 1 -- start start 2024-04-25T15:01:58.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d6cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d68076910 0x7f5d6819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d6cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68077b50 0x7f5d6819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d6cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d680784f0 0x7f5d681a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d6cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68113ed0 con 0x7f5d68077b50 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d6cc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68113d50 con 0x7f5d680784f0 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d6cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d68114050 con 0x7f5d68076910 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68077b50 0x7f5d6819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68077b50 0x7f5d6819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51762/0 (socket says 172.21.15.94:51762) 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 -- 172.21.15.94:0/3558625634 learned_addr learned my addr 172.21.15.94:0/3558625634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d67400640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d68076910 0x7f5d6819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d67e00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d680784f0 0x7f5d681a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 -- 172.21.15.94:0/3558625634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d68076910 msgr2=0x7f5d6819e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d68076910 0x7f5d6819e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 -- 172.21.15.94:0/3558625634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d680784f0 msgr2=0x7f5d681a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d680784f0 0x7f5d681a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.010+0000 7f5d66a00640 1 -- 172.21.15.94:0/3558625634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d681a6220 con 0x7f5d68077b50 2024-04-25T15:01:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.011+0000 7f5d66a00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68077b50 0x7f5d6819eb40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400e9f0 tx=0x7f5d5400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.011+0000 7f5d5fe00640 1 -- 172.21.15.94:0/3558625634 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400ccc0 con 0x7f5d68077b50 2024-04-25T15:01:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.011+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d681a6510 con 0x7f5d68077b50 2024-04-25T15:01:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.011+0000 7f5d5fe00640 1 -- 172.21.15.94:0/3558625634 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d540108e0 con 0x7f5d68077b50 2024-04-25T15:01:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.011+0000 7f5d5fe00640 1 -- 172.21.15.94:0/3558625634 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d54018880 con 0x7f5d68077b50 2024-04-25T15:01:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.011+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d681a6a60 con 0x7f5d68077b50 2024-04-25T15:01:58.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.012+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d68110bd0 con 0x7f5d68077b50 2024-04-25T15:01:58.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.015+0000 7f5d5fe00640 1 -- 172.21.15.94:0/3558625634 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5d54010430 con 0x7f5d68077b50 2024-04-25T15:01:58.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.016+0000 7f5d5fe00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.016+0000 7f5d5fe00640 1 -- 172.21.15.94:0/3558625634 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f5d540d07e0 con 0x7f5d68077b50 2024-04-25T15:01:58.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.016+0000 7f5d67400640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.017+0000 7f5d67400640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d580314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.021+0000 7f5d5fe00640 1 -- 172.21.15.94:0/3558625634 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d5409d180 con 0x7f5d68077b50 2024-04-25T15:01:58.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.261+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d68062b50 con 0x7f5d68077b50 2024-04-25T15:01:58.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.263+0000 7f5d5fe00640 1 -- 172.21.15.94:0/3558625634 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f5d5409ca80 con 0x7f5d68077b50 2024-04-25T15:01:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:58.264 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:01:58.265 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d4c0a3bb0 msgr2=0x7f5d4c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5d58000c00 tx=0x7f5d580314a0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68077b50 msgr2=0x7f5d6819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68077b50 0x7f5d6819eb40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400e9f0 tx=0x7f5d5400edf0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 shutdown_connections 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f5d4c0a3bb0 0x7f5d4c0a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5d680784f0 0x7f5d681a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5d68077b50 0x7f5d6819eb40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 --2- 172.21.15.94:0/3558625634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d68076910 0x7f5d6819e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.265+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 >> 172.21.15.94:0/3558625634 conn(0x7f5d680ff560 msgr2=0x7f5d68100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.266+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 shutdown_connections 2024-04-25T15:01:58.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.266+0000 7f5d6cc00640 1 -- 172.21.15.94:0/3558625634 wait complete. 2024-04-25T15:01:58.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:01:58.354 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:58 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4167835909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.354 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:58 smithi094 ceph-mon[27862]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:58.354 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:58 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1317554613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:58 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4167835909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:58 smithi138 ceph-mon[32875]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:58.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:58 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1317554613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:58 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4167835909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:58 smithi137 ceph-mon[32439]: pgmap v376: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:01:58.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:58 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1317554613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:01:58.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 -- 172.21.15.94:0/691484642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 msgr2=0x7f07501089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 --2- 172.21.15.94:0/691484642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f07501089d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0740009900 tx=0x7f074002a410 comp rx=0 tx=0).stop 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 -- 172.21.15.94:0/691484642 shutdown_connections 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 --2- 172.21.15.94:0/691484642 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f075010a160 0x7f07501109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 --2- 172.21.15.94:0/691484642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f07501097a0 0x7f0750109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 --2- 172.21.15.94:0/691484642 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f07501089d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 -- 172.21.15.94:0/691484642 >> 172.21.15.94:0/691484642 conn(0x7f0750077570 msgr2=0x7f0750079990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 -- 172.21.15.94:0/691484642 shutdown_connections 2024-04-25T15:01:58.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.483+0000 7f0757c00640 1 -- 172.21.15.94:0/691484642 wait complete. 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.484+0000 7f0757c00640 1 Processor -- start 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.484+0000 7f0757c00640 1 -- start start 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.484+0000 7f0757c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f075019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0757c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501097a0 0x7f075019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0757c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075010a160 0x7f07501a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0757c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0750114120 con 0x7f07501085b0 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0757c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0750113fa0 con 0x7f07501097a0 2024-04-25T15:01:58.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0757c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f07501142a0 con 0x7f075010a160 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f075019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f075019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51772/0 (socket says 172.21.15.94:51772) 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 -- 172.21.15.94:0/381597701 learned_addr learned my addr 172.21.15.94:0/381597701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755c00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075010a160 0x7f07501a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0747e00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501097a0 0x7f075019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 -- 172.21.15.94:0/381597701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075010a160 msgr2=0x7f07501a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075010a160 0x7f07501a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 -- 172.21.15.94:0/381597701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501097a0 msgr2=0x7f075019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501097a0 0x7f075019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.485+0000 7f0755200640 1 -- 172.21.15.94:0/381597701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07501a6440 con 0x7f07501085b0 2024-04-25T15:01:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.486+0000 7f0755200640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f075019e820 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0740031500 tx=0x7f0740003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.486+0000 7f074e000640 1 -- 172.21.15.94:0/381597701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074003f070 con 0x7f07501085b0 2024-04-25T15:01:58.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.486+0000 7f074e000640 1 -- 172.21.15.94:0/381597701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0740031c50 con 0x7f07501085b0 2024-04-25T15:01:58.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.486+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07501a66d0 con 0x7f07501085b0 2024-04-25T15:01:58.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.486+0000 7f074e000640 1 -- 172.21.15.94:0/381597701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0740035030 con 0x7f07501085b0 2024-04-25T15:01:58.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.486+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a6bc0 con 0x7f07501085b0 2024-04-25T15:01:58.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.487+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075010f940 con 0x7f07501085b0 2024-04-25T15:01:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.489+0000 7f074e000640 1 -- 172.21.15.94:0/381597701 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0740042070 con 0x7f07501085b0 2024-04-25T15:01:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.490+0000 7f074e000640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f07340a3bb0 0x7f07340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.490+0000 7f074e000640 1 -- 172.21.15.94:0/381597701 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f07400f0b00 con 0x7f07501085b0 2024-04-25T15:01:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.490+0000 7f0747e00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f07340a3bb0 0x7f07340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.491+0000 7f0747e00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f07340a3bb0 0x7f07340a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f075019fd30 tx=0x7f0738005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.494+0000 7f074e000640 1 -- 172.21.15.94:0/381597701 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07400bce90 con 0x7f07501085b0 2024-04-25T15:01:58.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.697+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0750062b50 con 0x7f07501085b0 2024-04-25T15:01:58.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.700+0000 7f074e000640 1 -- 172.21.15.94:0/381597701 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22684 (secure 0 0 0) 0x7f07400c1190 con 0x7f07501085b0 2024-04-25T15:01:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:01:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":88,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:00:02.693179+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f07340a3bb0 msgr2=0x7f07340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f07340a3bb0 0x7f07340a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f075019fd30 tx=0x7f0738005f90 comp rx=0 tx=0).stop 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 msgr2=0x7f075019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f075019e820 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0740031500 tx=0x7f0740003c10 comp rx=0 tx=0).stop 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 shutdown_connections 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f07340a3bb0 0x7f07340a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075010a160 0x7f07501a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f07501097a0 0x7f075019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 --2- 172.21.15.94:0/381597701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07501085b0 0x7f075019e820 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 >> 172.21.15.94:0/381597701 conn(0x7f0750077570 msgr2=0x7f075010cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 shutdown_connections 2024-04-25T15:01:58.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.703+0000 7f0757c00640 1 -- 172.21.15.94:0/381597701 wait complete. 2024-04-25T15:01:58.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T14:59:45.806095+0000', 'modified': '2024-04-25T14:59:50.088429+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24596}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24566': {'gid': 24566, 'name': 'g', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.94:6839/3586316280', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6838', 'nonce': 3586316280}, {'type': 'v1', 'addr': '172.21.15.94:6839', 'nonce': 3586316280}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24596': {'gid': 24596, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.137:6839/3444396688', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.137:6838', 'nonce': 3444396688}, {'type': 'v1', 'addr': '172.21.15.137:6839', 'nonce': 3444396688}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24596, 'qdb_cluster': [24596]} 2024-04-25T15:01:58.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T15:01:58.725 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T15:01:58.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 -- 172.21.15.94:0/4015415521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26841010b0 msgr2=0x7f2684101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 --2- 172.21.15.94:0/4015415521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26841010b0 0x7f2684101510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f266c009860 tx=0x7f266c02a3d0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 -- 172.21.15.94:0/4015415521 shutdown_connections 2024-04-25T15:01:58.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 --2- 172.21.15.94:0/4015415521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 0x7f2684101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 --2- 172.21.15.94:0/4015415521 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f26841010b0 0x7f2684101510 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 --2- 172.21.15.94:0/4015415521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2684100750 0x7f2684100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 -- 172.21.15.94:0/4015415521 >> 172.21.15.94:0/4015415521 conn(0x7f26840fc2f0 msgr2=0x7f26840fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 -- 172.21.15.94:0/4015415521 shutdown_connections 2024-04-25T15:01:58.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.924+0000 7f2689200640 1 -- 172.21.15.94:0/4015415521 wait complete. 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.925+0000 7f2689200640 1 Processor -- start 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2689200640 1 -- start start 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2689200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2684100750 0x7f268419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2689200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26841010b0 0x7f268419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2689200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 0x7f26841a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2689200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f268410fbf0 con 0x7f2684101a50 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2689200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f268410fa70 con 0x7f2684100750 2024-04-25T15:01:58.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2689200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f268410fd70 con 0x7f26841010b0 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2683e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 0x7f26841a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2683400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2684100750 0x7f268419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2682a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26841010b0 0x7f268419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2683400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2684100750 0x7f268419e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46018/0 (socket says 172.21.15.94:46018) 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2683400640 1 -- 172.21.15.94:0/339436375 learned_addr learned my addr 172.21.15.94:0/339436375 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2683e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 0x7f26841a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51794/0 (socket says 172.21.15.94:51794) 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.926+0000 7f2682a00640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26841010b0 0x7f268419ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60208/0 (socket says 172.21.15.94:60208) 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f2683400640 1 -- 172.21.15.94:0/339436375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26841010b0 msgr2=0x7f268419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f2683400640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26841010b0 0x7f268419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f2683400640 1 -- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 msgr2=0x7f26841a5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f2683400640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 0x7f26841a5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f2683400640 1 -- 172.21.15.94:0/339436375 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26841a6310 con 0x7f2684100750 2024-04-25T15:01:58.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f2683e00640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 0x7f26841a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:58.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f2683400640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2684100750 0x7f268419e6f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f267400ebc0 tx=0x7f267400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f267be00640 1 -- 172.21.15.94:0/339436375 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674010630 con 0x7f2684100750 2024-04-25T15:01:58.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.927+0000 7f267be00640 1 -- 172.21.15.94:0/339436375 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2674010c80 con 0x7f2684100750 2024-04-25T15:01:58.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.928+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26841a6600 con 0x7f2684100750 2024-04-25T15:01:58.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.928+0000 7f267be00640 1 -- 172.21.15.94:0/339436375 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674018b60 con 0x7f2684100750 2024-04-25T15:01:58.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.928+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26841a6b70 con 0x7f2684100750 2024-04-25T15:01:58.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 Processor -- start 2024-04-25T15:01:58.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 -- start start 2024-04-25T15:01:58.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d4b40 0x560cef8d9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d9cb0 0x560cef8dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef838b10 con 0x560cef8d4b40 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560cef7df2b0 con 0x560cef8d4150 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f268d7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cef834ad0 con 0x560cef8d9cb0 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f267be00640 1 -- 172.21.15.94:0/339436375 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2674018d00 con 0x7f2684100750 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f2683e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f2683e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46020/0 (socket says 172.21.15.94:46020) 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f2683e00640 1 -- 172.21.15.94:0/762250523 learned_addr learned my addr 172.21.15.94:0/762250523 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.931+0000 7f2682a00640 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d9cb0 0x560cef8dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2683400640 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d4b40 0x560cef8d9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2683e00640 1 -- 172.21.15.94:0/762250523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d9cb0 msgr2=0x560cef8dc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2683e00640 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d9cb0 0x560cef8dc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2683e00640 1 -- 172.21.15.94:0/762250523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d4b40 msgr2=0x560cef8d9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2683e00640 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d4b40 0x560cef8d9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2683e00640 1 -- 172.21.15.94:0/762250523 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cef54b2d0 con 0x560cef8d4150 2024-04-25T15:01:58.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2683e00640 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d4570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fb0 tx=0x7f267c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b465d323407cf8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2678c00640 1 -- 172.21.15.94:0/762250523 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01e070 con 0x560cef8d4150 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.932+0000 7f2678c00640 1 -- 172.21.15.94:0/762250523 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c0035f0 con 0x560cef8d4150 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f2678c00640 1 -- 172.21.15.94:0/762250523 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c00f800 con 0x560cef8d4150 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 -- 172.21.15.94:0/762250523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 msgr2=0x560cef8d4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d4570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f267c009fb0 tx=0x7f267c0118d0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f267be00640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f26680a3c30 0x7f26680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 -- 172.21.15.94:0/762250523 shutdown_connections 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d9cb0 0x560cef8dc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d4b40 0x560cef8d9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 --2- 172.21.15.94:0/762250523 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d4570 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 -- 172.21.15.94:0/762250523 >> 172.21.15.94:0/762250523 conn(0x560cef8cf7c0 msgr2=0x560cef8cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f267be00640 1 -- 172.21.15.94:0/339436375 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f26740d0940 con 0x7f2684100750 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f2683e00640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f26680a3c30 0x7f26680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.933+0000 7f268d7e5740 1 -- 172.21.15.94:0/762250523 shutdown_connections 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.934+0000 7f268d7e5740 1 -- 172.21.15.94:0/762250523 wait complete. 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.934+0000 7f268d7e5740 1 -- start start 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.934+0000 7f2683e00640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f26680a3c30 0x7f26680a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f267c011dd0 tx=0x7f267c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.934+0000 7f268d7e5740 1 Processor -- start 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f268d7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef8dcc60 con 0x560cef8d9cb0 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f268d7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560cef7f2570 con 0x560cef8d4150 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f268d7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cef8dcde0 con 0x560cef8d4b40 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f268d7e5740 10 client.0 _ll_register_callbacks cb 0x560cef7c0fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51818/0 (socket says 172.21.15.94:51818) 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 -- 172.21.15.94:0/3250295844 learned_addr learned my addr 172.21.15.94:0/3250295844 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683400640 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2682a00640 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 -- 172.21.15.94:0/3250295844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 msgr2=0x560cef8de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 -- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 msgr2=0x560cef8de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8de310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.935+0000 7f2683e00640 1 -- 172.21.15.94:0/3250295844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cef6156b0 con 0x560cef8d9cb0 2024-04-25T15:01:58.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f2683e00640 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e5890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f267c01b000 tx=0x7f267c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33409acfdfe5e7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f2671600640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f2671600640 1 -- 172.21.15.94:0/3250295844 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c034060 con 0x560cef8d9cb0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f2671600640 1 -- 172.21.15.94:0/3250295844 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f267c016040 con 0x560cef8d9cb0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c0035f0 con 0x560cef8d9cb0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f2671600640 1 -- 172.21.15.94:0/3250295844 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f267c01e040 con 0x560cef8d9cb0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f268d7e5740 10 client.14997 fetch_fsmap learned FSMap version 19 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.936+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560cef8e8c70 con 0x560cef8d9cb0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f2671600640 1 -- 172.21.15.94:0/3250295844 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c0035f0 con 0x560cef8d9cb0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f2671600640 1 -- 172.21.15.94:0/3250295844 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f267c039070 con 0x560cef8d9cb0 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f268d7e5740 10 client.14997 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f268d7e5740 10 client.14997 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f268d7e5740 20 client.14997 populate_metadata read hostname 'smithi094' 2024-04-25T15:01:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef8eb050 0x560cef8ec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f268d7e5740 4 client.14997 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T15:01:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 --> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cef846530 con 0x560cef8eb050 2024-04-25T15:01:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.937+0000 7f2683400640 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef8eb050 0x560cef8ec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.938+0000 7f2683400640 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef8eb050 0x560cef8ec4c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcbca0da94da65b0 server_cookie=142db2ce37b5b18b in_seq=0 out_seq=0 2024-04-25T15:01:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.938+0000 7f2671600640 10 client.14997 ms_handle_connect on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f2671600640 1 -- 172.21.15.94:0/3250295844 <== mds.0 v2:172.21.15.137:6838/3444396688 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560cef846530 con 0x560cef8eb050 2024-04-25T15:01:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f2671600640 10 client.14997 handle_command_reply: tid=0 2024-04-25T15:01:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 client.14997 shutdown 2024-04-25T15:01:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 msgr2=0x560cef8e5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e5890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f267c01b000 tx=0x7f267c0118d0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 shutdown_connections 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e5890 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef8eb050 0x560cef8ec4c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f2671600640 0 client.14997 ms_handle_reset on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3250295844 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 >> 172.21.15.94:0/3250295844 conn(0x560cef8cf7c0 msgr2=0x560cef8cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 shutdown_connections 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 1 -- 172.21.15.94:0/3250295844 wait complete. 2024-04-25T15:01:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.939+0000 7f268d7e5740 20 client.14997 trim_cache size 0 max 16384 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 Processor -- start 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 -- start start 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef8ea800 con 0x560cef8d9cb0 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560cef8eaed0 con 0x560cef8d4150 2024-04-25T15:01:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f268d7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cef8eb5a0 con 0x560cef8d4b40 2024-04-25T15:01:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f2682a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.959+0000 7f2683e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8e5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60230/0 (socket says 172.21.15.94:60230) 2024-04-25T15:01:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 -- 172.21.15.94:0/3986423179 learned_addr learned my addr 172.21.15.94:0/3986423179 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2683400640 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 -- 172.21.15.94:0/3986423179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 msgr2=0x560cef8d92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 -- 172.21.15.94:0/3986423179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 msgr2=0x560cef8d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8d5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 -- 172.21.15.94:0/3986423179 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cef8dcf80 con 0x560cef8d4b40 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2683e00640 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2683400640 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.960+0000 7f2682a00640 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8e5850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f266c02a8f0 tx=0x7f266c030980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6003a1b09ccbb940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f2673e00640 1 -- 172.21.15.94:0/3986423179 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c04c070 con 0x560cef8d4b40 2024-04-25T15:01:58.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f2673e00640 1 -- 172.21.15.94:0/3986423179 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0054f0 con 0x560cef8d4b40 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f2673e00640 1 -- 172.21.15.94:0/3986423179 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c004c80 con 0x560cef8d4b40 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 -- 172.21.15.94:0/3986423179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 msgr2=0x560cef8e5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8e5850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f266c02a8f0 tx=0x7f266c030980 comp rx=0 tx=0).stop 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 -- 172.21.15.94:0/3986423179 shutdown_connections 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4b40 0x560cef8e5850 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 --2- 172.21.15.94:0/3986423179 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4150 0x560cef8d92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 -- 172.21.15.94:0/3986423179 >> 172.21.15.94:0/3986423179 conn(0x560cef8cf7c0 msgr2=0x560cef8ea020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 -- 172.21.15.94:0/3986423179 shutdown_connections 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.961+0000 7f268d7e5740 1 -- 172.21.15.94:0/3986423179 wait complete. 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 -- start start 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 Processor -- start 2024-04-25T15:01:58.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4150 0x560cef8e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4b40 0x560cef8e1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x560cef8ec340 con 0x560cef8d9cb0 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x560cef8ebc70 con 0x560cef8d4b40 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560cef8eca10 con 0x560cef8d4150 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.962+0000 7f268d7e5740 10 client.0 _ll_register_callbacks cb 0x7f267c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4150 0x560cef8e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2682a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4b40 0x560cef8e1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4150 0x560cef8e1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60232/0 (socket says 172.21.15.94:60232) 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 -- 172.21.15.94:0/2710951299 learned_addr learned my addr 172.21.15.94:0/2710951299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683e00640 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 -- 172.21.15.94:0/2710951299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4b40 msgr2=0x560cef8e1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4b40 0x560cef8e1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 -- 172.21.15.94:0/2710951299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 msgr2=0x560cef8e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 -- 172.21.15.94:0/2710951299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cef8e4fc0 con 0x560cef8d4150 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683e00640 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2683400640 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4150 0x560cef8e1aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ae0 tx=0x7f2674018600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92afb9bfd2db5382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2671600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.963+0000 7f2671600640 1 -- 172.21.15.94:0/2710951299 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740a0fd0 con 0x560cef8d4150 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f2671600640 1 -- 172.21.15.94:0/2710951299 <== mon.1 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f2674099340 con 0x560cef8d4150 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266c0054f0 con 0x560cef8d4150 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f2671600640 1 -- 172.21.15.94:0/2710951299 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f26740a5890 con 0x560cef8d4150 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 10 client.25117 fetch_fsmap learned FSMap version 19 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x560cef951480 con 0x560cef8d4150 2024-04-25T15:01:58.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f2671600640 1 -- 172.21.15.94:0/2710951299 <== mon.1 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266c0054f0 con 0x560cef8d4150 2024-04-25T15:01:58.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f2671600640 1 -- 172.21.15.94:0/2710951299 <== mon.1 v2:172.21.15.138:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f2674098a20 con 0x560cef8d4150 2024-04-25T15:01:58.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 10 client.25117 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T15:01:58.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 10 client.25117 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T15:01:58.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 20 client.25117 populate_metadata read hostname 'smithi094' 2024-04-25T15:01:58.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef956010 0x560cef954d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:58.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 4 client.25117 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T15:01:58.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.964+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 --> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] -- command(tid 0: {"prefix": "status"}) -- 0x560cef8403a0 con 0x560cef956010 2024-04-25T15:01:58.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.965+0000 7f2682a00640 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef956010 0x560cef954d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:58.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f2682a00640 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef956010 0x560cef954d20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc0b17c948759d7 server_cookie=1017e026de9b126f in_seq=0 out_seq=0 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f2671600640 10 client.25117 ms_handle_connect on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f2671600640 1 -- 172.21.15.94:0/2710951299 <== mds.0 v2:172.21.15.137:6838/3444396688 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560cef8403a0 con 0x560cef956010 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f2671600640 10 client.25117 handle_command_reply: tid=0 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 client.25117 shutdown 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4150 msgr2=0x560cef8e1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4150 0x560cef8e1aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2674009ae0 tx=0x7f2674018600 comp rx=0 tx=0).stop 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 shutdown_connections 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560cef8d9cb0 0x560cef8e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x560cef8d4b40 0x560cef8e1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x560cef956010 0x560cef954d20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 --2- 172.21.15.94:0/2710951299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560cef8d4150 0x560cef8e1aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 >> 172.21.15.94:0/2710951299 conn(0x560cef8cf7c0 msgr2=0x560cef8de310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f2671600640 0 client.25117 ms_handle_reset on v2:172.21.15.137:6838/3444396688 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 shutdown_connections 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 1 -- 172.21.15.94:0/2710951299 wait complete. 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.966+0000 7f268d7e5740 20 client.25117 trim_cache size 0 max 16384 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "f04d929c-0312-11ef-bc93-c7b262605968", 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24596, 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-04-25T15:01:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 132.39529488400001, 2024-04-25T15:01:58.974 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 19, 2024-04-25T15:01:58.974 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 88, 2024-04-25T15:01:58.974 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T15:01:58.974 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 138.847812637 2024-04-25T15:01:58.974 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f26680a3c30 msgr2=0x7f26680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f26680a3c30 0x7f26680a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f267c011dd0 tx=0x7f267c000bc0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2684100750 msgr2=0x7f268419e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2684100750 0x7f268419e6f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f267400ebc0 tx=0x7f267400c6a0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 shutdown_connections 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f26680a3c30 0x7f26680a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2684101a50 0x7f26841a5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26841010b0 0x7f268419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 --2- 172.21.15.94:0/339436375 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2684100750 0x7f268419e6f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:58.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 >> 172.21.15.94:0/339436375 conn(0x7f26840fc2f0 msgr2=0x7f26840fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:58.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 shutdown_connections 2024-04-25T15:01:58.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:58.967+0000 7f2689200640 1 -- 172.21.15.94:0/339436375 wait complete. 2024-04-25T15:01:58.987 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T15:01:58.997 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T15:01:58.997 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T15:01:58.997 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T15:01:58.997 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T15:01:58.998 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T15:01:58.998 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:01:58.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T15:01:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.193+0000 7f7229a00640 1 -- 172.21.15.94:0/2546903110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224077b50 msgr2=0x7f7224077fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.193+0000 7f7229a00640 1 --2- 172.21.15.94:0/2546903110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224077b50 0x7f7224077fb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7210009860 tx=0x7f721002a340 comp rx=0 tx=0).stop 2024-04-25T15:01:59.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.194+0000 7f7229a00640 1 -- 172.21.15.94:0/2546903110 shutdown_connections 2024-04-25T15:01:59.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.194+0000 7f7229a00640 1 --2- 172.21.15.94:0/2546903110 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72240784f0 0x7f7224078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.194+0000 7f7229a00640 1 --2- 172.21.15.94:0/2546903110 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224077b50 0x7f7224077fb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.194+0000 7f7229a00640 1 --2- 172.21.15.94:0/2546903110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7224076910 0x7f7224076d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.194+0000 7f7229a00640 1 -- 172.21.15.94:0/2546903110 >> 172.21.15.94:0/2546903110 conn(0x7f72240ff740 msgr2=0x7f7224101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:01:59.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.194+0000 7f7229a00640 1 -- 172.21.15.94:0/2546903110 shutdown_connections 2024-04-25T15:01:59.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.194+0000 7f7229a00640 1 -- 172.21.15.94:0/2546903110 wait complete. 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 Processor -- start 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 -- start start 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224076910 0x7f722419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7224077b50 0x7f722419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72240784f0 0x7f72241a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f72241140d0 con 0x7f7224076910 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7224113f50 con 0x7f72240784f0 2024-04-25T15:01:59.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7229a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7224114250 con 0x7f7224077b50 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7222a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224076910 0x7f722419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7222a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224076910 0x7f722419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51844/0 (socket says 172.21.15.94:51844) 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7222a00640 1 -- 172.21.15.94:0/18599065 learned_addr learned my addr 172.21.15.94:0/18599065 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.195+0000 7f7222000640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7224077b50 0x7f722419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7223400640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72240784f0 0x7f72241a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7222a00640 1 -- 172.21.15.94:0/18599065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7224077b50 msgr2=0x7f722419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7222a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7224077b50 0x7f722419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7222a00640 1 -- 172.21.15.94:0/18599065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72240784f0 msgr2=0x7f72241a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7222a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72240784f0 0x7f72241a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7222a00640 1 -- 172.21.15.94:0/18599065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72241a6360 con 0x7f7224076910 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7223400640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72240784f0 0x7f72241a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7222a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224076910 0x7f722419e740 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f721400e9c0 tx=0x7f721400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:59.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f721b400640 1 -- 172.21.15.94:0/18599065 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721400ccc0 con 0x7f7224076910 2024-04-25T15:01:59.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f721b400640 1 -- 172.21.15.94:0/18599065 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72140054d0 con 0x7f7224076910 2024-04-25T15:01:59.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f721b400640 1 -- 172.21.15.94:0/18599065 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7214010430 con 0x7f7224076910 2024-04-25T15:01:59.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.196+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72241a6650 con 0x7f7224076910 2024-04-25T15:01:59.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.197+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72241a6b70 con 0x7f7224076910 2024-04-25T15:01:59.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.198+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7224110b40 con 0x7f7224076910 2024-04-25T15:01:59.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.200+0000 7f721b400640 1 -- 172.21.15.94:0/18599065 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7214004f80 con 0x7f7224076910 2024-04-25T15:01:59.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.201+0000 7f721b400640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f72080a3c50 0x7f72080a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:01:59.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.201+0000 7f721b400640 1 -- 172.21.15.94:0/18599065 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f72140d0910 con 0x7f7224076910 2024-04-25T15:01:59.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.201+0000 7f7222000640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f72080a3c50 0x7f72080a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:01:59.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.202+0000 7f7222000640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f72080a3c50 0x7f72080a6110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f721002a930 tx=0x7f7210030450 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:01:59.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.205+0000 7f721b400640 1 -- 172.21.15.94:0/18599065 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f721409d2b0 con 0x7f7224076910 2024-04-25T15:01:59.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:59 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3558625634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:59.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:01:59 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/381597701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:59.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:01:59.460+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f7224062b50 con 0x7f7224076910 2024-04-25T15:01:59.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:59 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3558625634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:59.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:01:59 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/381597701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:01:59.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:59 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3558625634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:01:59.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:01:59 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/381597701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:00.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.122+0000 7f721b400640 1 -- 172.21.15.94:0/18599065 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f721409cbb0 con 0x7f7224076910 2024-04-25T15:02:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.125+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f72080a3c50 msgr2=0x7f72080a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.125+0000 7f7229a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f72080a3c50 0x7f72080a6110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f721002a930 tx=0x7f7210030450 comp rx=0 tx=0).stop 2024-04-25T15:02:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.125+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224076910 msgr2=0x7f722419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.125+0000 7f7229a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224076910 0x7f722419e740 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f721400e9c0 tx=0x7f721400ee80 comp rx=0 tx=0).stop 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 shutdown_connections 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f72080a3c50 0x7f72080a6110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f72240784f0 0x7f72241a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7224077b50 0x7f722419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 --2- 172.21.15.94:0/18599065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7224076910 0x7f722419e740 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 >> 172.21.15.94:0/18599065 conn(0x7f72240ff740 msgr2=0x7f7224101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 shutdown_connections 2024-04-25T15:02:00.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.126+0000 7f7229a00640 1 -- 172.21.15.94:0/18599065 wait complete. 2024-04-25T15:02:00.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T15:02:00.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 -- 172.21.15.94:0/629251130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 msgr2=0x7f185c103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 --2- 172.21.15.94:0/629251130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 0x7f185c103620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802a410 comp rx=0 tx=0).stop 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 -- 172.21.15.94:0/629251130 shutdown_connections 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 --2- 172.21.15.94:0/629251130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c103b60 0x7f185c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 --2- 172.21.15.94:0/629251130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 0x7f185c103620 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 --2- 172.21.15.94:0/629251130 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f185c101fb0 0x7f185c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 -- 172.21.15.94:0/629251130 >> 172.21.15.94:0/629251130 conn(0x7f185c0fd540 msgr2=0x7f185c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 -- 172.21.15.94:0/629251130 shutdown_connections 2024-04-25T15:02:00.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.334+0000 7f1863800640 1 -- 172.21.15.94:0/629251130 wait complete. 2024-04-25T15:02:00.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 Processor -- start 2024-04-25T15:02:00.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 -- start start 2024-04-25T15:02:00.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c101fb0 0x7f185c107d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 0x7f185c108270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f185c103b60 0x7f185c1048a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f185c10db40 con 0x7f185c1031a0 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f185c10d9c0 con 0x7f185c103b60 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1863800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f185c10dcc0 con 0x7f185c101fb0 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.335+0000 7f1860e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c101fb0 0x7f185c107d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f185be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 0x7f185c108270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c101fb0 0x7f185c107d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60266/0 (socket says 172.21.15.94:60266) 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 -- 172.21.15.94:0/1102905076 learned_addr learned my addr 172.21.15.94:0/1102905076 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1861800640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f185c103b60 0x7f185c1048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 -- 172.21.15.94:0/1102905076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f185c103b60 msgr2=0x7f185c1048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f185c103b60 0x7f185c1048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 -- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 msgr2=0x7f185c108270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 0x7f185c108270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 -- 172.21.15.94:0/1102905076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c105160 con 0x7f185c101fb0 2024-04-25T15:02:00.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1860e00640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c101fb0 0x7f185c107d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f185000e9c0 tx=0x7f185000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1859600640 1 -- 172.21.15.94:0/1102905076 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185000cde0 con 0x7f185c101fb0 2024-04-25T15:02:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.336+0000 7f1859600640 1 -- 172.21.15.94:0/1102905076 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18500108e0 con 0x7f185c101fb0 2024-04-25T15:02:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.337+0000 7f1859600640 1 -- 172.21.15.94:0/1102905076 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850018960 con 0x7f185c101fb0 2024-04-25T15:02:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.337+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c1a6740 con 0x7f185c101fb0 2024-04-25T15:02:00.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.337+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c1a6af0 con 0x7f185c101fb0 2024-04-25T15:02:00.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.338+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c04fce0 con 0x7f185c101fb0 2024-04-25T15:02:00.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.340+0000 7f1859600640 1 -- 172.21.15.94:0/1102905076 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f185001f020 con 0x7f185c101fb0 2024-04-25T15:02:00.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.341+0000 7f1859600640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 0x7f18440a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.341+0000 7f185be00640 1 -- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 msgr2=0x7f18440a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:00.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.341+0000 7f1859600640 1 -- 172.21.15.94:0/1102905076 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f18500d0710 con 0x7f185c101fb0 2024-04-25T15:02:00.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.341+0000 7f185be00640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 0x7f18440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:02:00.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.345+0000 7f1859600640 1 -- 172.21.15.94:0/1102905076 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1850053ff0 con 0x7f185c101fb0 2024-04-25T15:02:00.446 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: ignoring --setuser ceph since I am not root 2024-04-25T15:02:00.446 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: ignoring --setgroup ceph since I am not root 2024-04-25T15:02:00.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:00 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/18599065' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T15:02:00.446 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:00 smithi094 ceph-mon[27862]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:00.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/18599065' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T15:02:00.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-mon[32875]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:00.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: ignoring --setuser ceph since I am not root 2024-04-25T15:02:00.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: ignoring --setgroup ceph since I am not root 2024-04-25T15:02:00.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-mgr[34734]: -- 172.21.15.138:0/818163695 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f2037645a0 con 0x55f203738800 2024-04-25T15:02:00.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:00.309+0000 7f22dc7f5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T15:02:00.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/18599065' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T15:02:00.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-mon[32439]: pgmap v377: 97 pgs: 97 active+clean; 637 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:00.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: ignoring --setuser ceph since I am not root 2024-04-25T15:02:00.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: ignoring --setgroup ceph since I am not root 2024-04-25T15:02:00.483 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-mgr[33832]: -- 172.21.15.137:0/229496137 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d404f665a0 con 0x55d404f3b000 2024-04-25T15:02:00.483 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:00.308+0000 7f5a8bbe0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T15:02:00.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.542+0000 7f185be00640 1 -- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 msgr2=0x7f18440a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:00.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.542+0000 7f185be00640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 0x7f18440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:02:00.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.549+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f185c1a7490 con 0x7f185c101fb0 2024-04-25T15:02:00.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.557+0000 7f1859600640 1 -- 172.21.15.94:0/1102905076 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f185009d0b0 con 0x7f185c101fb0 2024-04-25T15:02:00.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 msgr2=0x7f18440a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:02:00.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 0x7f18440a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c101fb0 msgr2=0x7f185c107d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c101fb0 0x7f185c107d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f185000e9c0 tx=0x7f185000ee80 comp rx=0 tx=0).stop 2024-04-25T15:02:00.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 shutdown_connections 2024-04-25T15:02:00.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f18440a3ca0 0x7f18440a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f185c103b60 0x7f185c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f185c1031a0 0x7f185c108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 --2- 172.21.15.94:0/1102905076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f185c101fb0 0x7f185c107d30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.560+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 >> 172.21.15.94:0/1102905076 conn(0x7f185c0fd540 msgr2=0x7f185c074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:00.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.561+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 shutdown_connections 2024-04-25T15:02:00.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.561+0000 7f1863800640 1 -- 172.21.15.94:0/1102905076 wait complete. 2024-04-25T15:02:00.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T15:02:00.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:00.444+0000 7ff6a5c8b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T15:02:00.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:00.691+0000 7ff6a5c8b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:02:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 -- 172.21.15.94:0/1296424169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288102fd0 msgr2=0x7f8288103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 --2- 172.21.15.94:0/1296424169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288102fd0 0x7f8288103450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8274009860 tx=0x7f827402a340 comp rx=0 tx=0).stop 2024-04-25T15:02:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 -- 172.21.15.94:0/1296424169 shutdown_connections 2024-04-25T15:02:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 --2- 172.21.15.94:0/1296424169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288103990 0x7f828810a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 --2- 172.21.15.94:0/1296424169 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288102fd0 0x7f8288103450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 --2- 172.21.15.94:0/1296424169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288101de0 0x7f8288102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 -- 172.21.15.94:0/1296424169 >> 172.21.15.94:0/1296424169 conn(0x7f82880fd350 msgr2=0x7f82880ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:00.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 -- 172.21.15.94:0/1296424169 shutdown_connections 2024-04-25T15:02:00.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.756+0000 7f828cc00640 1 -- 172.21.15.94:0/1296424169 wait complete. 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.757+0000 7f828cc00640 1 Processor -- start 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.757+0000 7f828cc00640 1 -- start start 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.757+0000 7f828cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288101de0 0x7f8288075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.757+0000 7f828cc00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288102fd0 0x7f8288074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.757+0000 7f828cc00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288103990 0x7f8288074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.757+0000 7f828cc00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f828810d8f0 con 0x7f8288102fd0 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f828cc00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f828810d770 con 0x7f8288103990 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f828cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f828810da70 con 0x7f8288101de0 2024-04-25T15:02:00.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288101de0 0x7f8288075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288101de0 0x7f8288075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:60284/0 (socket says 172.21.15.94:60284) 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 -- 172.21.15.94:0/2926204139 learned_addr learned my addr 172.21.15.94:0/2926204139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8286a00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288102fd0 0x7f8288074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287e00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288103990 0x7f8288074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 -- 172.21.15.94:0/2926204139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288103990 msgr2=0x7f8288074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288103990 0x7f8288074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 -- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288102fd0 msgr2=0x7f8288074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288102fd0 0x7f8288074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 -- 172.21.15.94:0/2926204139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8288075110 con 0x7f8288101de0 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287e00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288103990 0x7f8288074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.758+0000 7f8287400640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288101de0 0x7f8288075d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f827800e9a0 tx=0x7f827800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.759+0000 7f827fe00640 1 -- 172.21.15.94:0/2926204139 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827800cd80 con 0x7f8288101de0 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.759+0000 7f827fe00640 1 -- 172.21.15.94:0/2926204139 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82780108e0 con 0x7f8288101de0 2024-04-25T15:02:00.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.759+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82881a2100 con 0x7f8288101de0 2024-04-25T15:02:00.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.759+0000 7f827fe00640 1 -- 172.21.15.94:0/2926204139 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8278018960 con 0x7f8288101de0 2024-04-25T15:02:00.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.759+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82881a2620 con 0x7f8288101de0 2024-04-25T15:02:00.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.760+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8288050530 con 0x7f8288101de0 2024-04-25T15:02:00.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.762+0000 7f827fe00640 1 -- 172.21.15.94:0/2926204139 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f8278010a90 con 0x7f8288101de0 2024-04-25T15:02:00.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.763+0000 7f827fe00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 0x7f826c0a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:00.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.763+0000 7f827fe00640 1 -- 172.21.15.94:0/2926204139 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f82780d0880 con 0x7f8288101de0 2024-04-25T15:02:00.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.763+0000 7f8286a00640 1 -- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 msgr2=0x7f826c0a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:00.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.763+0000 7f8286a00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 0x7f826c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:02:00.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.767+0000 7f827fe00640 1 -- 172.21.15.94:0/2926204139 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8278053ff0 con 0x7f8288101de0 2024-04-25T15:02:00.907 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:00.497+0000 7f22dc7f5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:02:00.907 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:00.600+0000 7f22dc7f5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T15:02:00.907 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:00.905+0000 7f22dc7f5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T15:02:00.907 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:00.493+0000 7f5a8bbe0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:02:00.907 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:00.597+0000 7f5a8bbe0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T15:02:00.908 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:00.906+0000 7f5a8bbe0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T15:02:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.964+0000 7f8286a00640 1 -- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 msgr2=0x7f826c0a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:00.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.964+0000 7f8286a00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 0x7f826c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:02:00.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.974+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f82881091a0 con 0x7f8288101de0 2024-04-25T15:02:00.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.983+0000 7f827fe00640 1 -- 172.21.15.94:0/2926204139 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f827809d220 con 0x7f8288101de0 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 msgr2=0x7f826c0a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 0x7f826c0a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288101de0 msgr2=0x7f8288075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288101de0 0x7f8288075d00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f827800e9a0 tx=0x7f827800ee60 comp rx=0 tx=0).stop 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 shutdown_connections 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f826c0a3bf0 0x7f826c0a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f8288103990 0x7f8288074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8288102fd0 0x7f8288074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 --2- 172.21.15.94:0/2926204139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8288101de0 0x7f8288075d00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.985+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 >> 172.21.15.94:0/2926204139 conn(0x7f82880fd350 msgr2=0x7f8288106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:00.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.986+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 shutdown_connections 2024-04-25T15:02:00.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:00.986+0000 7f828cc00640 1 -- 172.21.15.94:0/2926204139 wait complete. 2024-04-25T15:02:01.005 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T15:02:01.042 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T15:02:01.042 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T15:02:01.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:01.043 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T15:02:01.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:01.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T15:02:01.126 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:00.829+0000 7ff6a5c8b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T15:02:01.207 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:00.991+0000 7f22dc7f5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T15:02:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:01 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/18599065' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T15:02:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:01 smithi138 ceph-mon[32875]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T15:02:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:01 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' 2024-04-25T15:02:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:01 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' 2024-04-25T15:02:01.208 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-04-25T15:02:01.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:02:01.210 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T15:02:01.210 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T15:02:01.210 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T15:02:01.211 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T15:02:01.232 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:00.995+0000 7f5a8bbe0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T15:02:01.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:01 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/18599065' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T15:02:01.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:01 smithi137 ceph-mon[32439]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T15:02:01.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:01 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' 2024-04-25T15:02:01.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:01 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' 2024-04-25T15:02:01.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:01 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/18599065' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T15:02:01.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:01 smithi094 ceph-mon[27862]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T15:02:01.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:01 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' 2024-04-25T15:02:01.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:01 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' 2024-04-25T15:02:01.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:01.238+0000 7ff6a5c8b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T15:02:01.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:01.356+0000 7ff6a5c8b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T15:02:01.621 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:02.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:02.176+0000 7f22dc7f5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T15:02:02.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:02.349+0000 7f22dc7f5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T15:02:02.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:02.179+0000 7f5a8bbe0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T15:02:02.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:02.351+0000 7f5a8bbe0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T15:02:02.813 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:02.520+0000 7f22dc7f5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T15:02:02.813 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:02.812+0000 7f22dc7f5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T15:02:02.815 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:02.523+0000 7f5a8bbe0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T15:02:03.110 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:02.905+0000 7f22dc7f5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T15:02:03.110 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:03.108+0000 7f22dc7f5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T15:02:03.113 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:02.814+0000 7f5a8bbe0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T15:02:03.113 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:02.907+0000 7f5a8bbe0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T15:02:03.113 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:03.112+0000 7f5a8bbe0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T15:02:03.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:02.938+0000 7ff6a5c8b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T15:02:03.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:03.173+0000 7ff6a5c8b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T15:02:03.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:03.304+0000 7f22dc7f5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:02:03.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:03.305+0000 7f5a8bbe0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:02:03.500 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:03.392+0000 7ff6a5c8b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.755+0000 7f0879e00700 1 -- 172.21.15.94:0/474734751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 msgr2=0x7f087410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.755+0000 7f0879e00700 1 --2- 172.21.15.94:0/474734751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 0x7f087410ef00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f085c009990 tx=0x7f085c009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.756+0000 7f0879e00700 1 -- 172.21.15.94:0/474734751 shutdown_connections 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.756+0000 7f0879e00700 1 --2- 172.21.15.94:0/474734751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f087410f440 0x7f0874111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.756+0000 7f0879e00700 1 --2- 172.21.15.94:0/474734751 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 0x7f0874075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.756+0000 7f0879e00700 1 --2- 172.21.15.94:0/474734751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 0x7f087410ef00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.756+0000 7f0879e00700 1 -- 172.21.15.94:0/474734751 >> 172.21.15.94:0/474734751 conn(0x7f0874079ba0 msgr2=0x7f0874079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:03.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.756+0000 7f0879e00700 1 -- 172.21.15.94:0/474734751 shutdown_connections 2024-04-25T15:02:03.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.756+0000 7f0879e00700 1 -- 172.21.15.94:0/474734751 wait complete. 2024-04-25T15:02:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.757+0000 7f0879e00700 1 Processor -- start 2024-04-25T15:02:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.757+0000 7f0879e00700 1 -- start start 2024-04-25T15:02:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0879e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 0x7f087419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0879e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 0x7f087419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0879e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f087410f440 0x7f087419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0879e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0874114e50 con 0x7f087410f440 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0879e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0874114cd0 con 0x7f0874074c70 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0879e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0874114fd0 con 0x7f0874075590 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0872a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 0x7f087419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0872a00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 0x7f087419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:46112/0 (socket says 172.21.15.94:46112) 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0872a00700 1 -- 172.21.15.94:0/1950042645 learned_addr learned my addr 172.21.15.94:0/1950042645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f086be00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 0x7f087419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0873400700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f087410f440 0x7f087419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.758+0000 7f0872a00700 1 -- 172.21.15.94:0/1950042645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 msgr2=0x7f087419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0872a00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 0x7f087419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0872a00700 1 -- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f087410f440 msgr2=0x7f087419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:03.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0872a00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f087410f440 0x7f087419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:03.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0872a00700 1 -- 172.21.15.94:0/1950042645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08741a0300 con 0x7f0874074c70 2024-04-25T15:02:03.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0872a00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 0x7f087419b340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f086400e9a0 tx=0x7f086400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:03.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0878a00700 1 -- 172.21.15.94:0/1950042645 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086400ca10 con 0x7f0874074c70 2024-04-25T15:02:03.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0878a00700 1 -- 172.21.15.94:0/1950042645 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f086400cbb0 con 0x7f0874074c70 2024-04-25T15:02:03.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08741a05f0 con 0x7f0874074c70 2024-04-25T15:02:03.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0878a00700 1 -- 172.21.15.94:0/1950042645 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864018680 con 0x7f0874074c70 2024-04-25T15:02:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.759+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08741a7e70 con 0x7f0874074c70 2024-04-25T15:02:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.760+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0874069730 con 0x7f0874074c70 2024-04-25T15:02:03.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.762+0000 7f0878a00700 1 -- 172.21.15.94:0/1950042645 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0864010430 con 0x7f0874074c70 2024-04-25T15:02:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.763+0000 7f0878a00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 0x7f08580a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.764+0000 7f086be00700 1 -- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 msgr2=0x7f08580a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.764+0000 7f0878a00700 1 -- 172.21.15.94:0/1950042645 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f08640d0620 con 0x7f0874074c70 2024-04-25T15:02:03.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.764+0000 7f086be00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 0x7f08580a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:02:03.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.767+0000 7f0878a00700 1 -- 172.21.15.94:0/1950042645 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f086409cfc0 con 0x7f0874074c70 2024-04-25T15:02:03.934 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:03.807+0000 7ff6a5c8b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T15:02:03.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.965+0000 7f086be00700 1 -- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 msgr2=0x7f08580a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:03.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:03.965+0000 7f086be00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 0x7f08580a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:02:04.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.044+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f087402e0a0 con 0x7f0874074c70 2024-04-25T15:02:04.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.044+0000 7f0878a00700 1 -- 172.21.15.94:0/1950042645 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7f086409c8c0 con 0x7f0874074c70 2024-04-25T15:02:04.046 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T15:02:04.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.047+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 msgr2=0x7f08580a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:02:04.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.047+0000 7f0879e00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 0x7f08580a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.047+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 msgr2=0x7f087419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:04.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.047+0000 7f0879e00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 0x7f087419b340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f086400e9a0 tx=0x7f086400ecb0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 shutdown_connections 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f08580a3c90 0x7f08580a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f087410f440 0x7f087419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0874074c70 0x7f087419b340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 --2- 172.21.15.94:0/1950042645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0874075590 0x7f087419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 >> 172.21.15.94:0/1950042645 conn(0x7f0874079ba0 msgr2=0x7f08740ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 shutdown_connections 2024-04-25T15:02:04.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04.048+0000 7f0879e00700 1 -- 172.21.15.94:0/1950042645 wait complete. 2024-04-25T15:02:04.198 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:04 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1950042645' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:02:04.199 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:03.932+0000 7ff6a5c8b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T15:02:04.206 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:03.905+0000 7f22dc7f5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T15:02:04.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:04 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1950042645' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:02:04.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:04 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1950042645' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:02:04.232 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:03.905+0000 7f5a8bbe0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T15:02:04.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:04.196+0000 7ff6a5c8b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T15:02:04.630 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:04.330+0000 7f5a8bbe0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T15:02:04.631 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:04.427+0000 7f5a8bbe0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T15:02:04.631 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:04.629+0000 7f5a8bbe0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T15:02:04.633 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:04.332+0000 7f22dc7f5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T15:02:04.634 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:04.429+0000 7f22dc7f5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T15:02:04.634 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:04.632+0000 7f22dc7f5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T15:02:04.726 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:04.463+0000 7ff6a5c8b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T15:02:04.784 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T15:02:04.784 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T15:02:04.785 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-04-25T15:02:04.785 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T15:02:04.785 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T15:02:04.785 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T15:02:04.785 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T15:02:04.785 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T15:02:04.785 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T15:02:04.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:04.785 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-04-25T15:02:04.828 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T15:02:04.828 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T15:02:04.828 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T15:02:04.828 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:02:04.828 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-04-25T15:02:04.828 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 41950sec preferred_lft 41950sec 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-04-25T15:02:04.829 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:02:04.831 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T15:02:04.831 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:02:04.831 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T15:02:04.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-04-25T15:02:04.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-04-25T15:02:04.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-04-25T15:02:04.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T15:02:04.831 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T15:02:04.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:02:05.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:05.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:05.331 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T15:02:05.394 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-04-25T15:02:05.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:05.398 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-04-25T15:02:05.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:05.159+0000 7f22dc7f5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T15:02:05.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:05.252+0000 7f22dc7f5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T15:02:05.459 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-04-25T15:02:05.459 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T15:02:05.459 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T15:02:05.459 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T15:02:05.459 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T15:02:05.461 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:02:05.461 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T15:02:05.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T15:02:05.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T15:02:05.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T15:02:05.461 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T15:02:05.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:05.156+0000 7f5a8bbe0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T15:02:05.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:05.248+0000 7f5a8bbe0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T15:02:05.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:02:05.571 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:05 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:05.295+0000 7ff6a5c8b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T15:02:05.837 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:05.492+0000 7f5a8bbe0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T15:02:05.837 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:05.624+0000 7f5a8bbe0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T15:02:05.837 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:05.719+0000 7f5a8bbe0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T15:02:05.837 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:05.836+0000 7f5a8bbe0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T15:02:05.839 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:05.497+0000 7f22dc7f5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T15:02:05.839 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:05.628+0000 7f22dc7f5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T15:02:05.839 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:05.722+0000 7f22dc7f5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T15:02:05.840 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:05.838+0000 7f22dc7f5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T15:02:06.175 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:05 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:05.885+0000 7ff6a5c8b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T15:02:06.175 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:06.017+0000 7ff6a5c8b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T15:02:06.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:06.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:06.186 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T15:02:06.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:06.229 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-04-25T15:02:06.313 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:02:06.313 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T15:02:06.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:02:06.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T15:02:06.314 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T15:02:06.457 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:06.142+0000 7f22dc7f5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T15:02:06.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:02:06.462 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:06.291+0000 7ff6a5c8b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T15:02:06.482 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:06.140+0000 7f5a8bbe0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T15:02:06.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:06.533 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T15:02:06.533 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:02:06.533 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T15:02:06.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T15:02:06.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T15:02:06.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T15:02:06.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T15:02:06.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T15:02:06.534 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T15:02:06.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:02:06.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:06.824 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:02:06.824 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-04-25T15:02:06.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-04-25T15:02:06.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T15:02:06.824 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-04-25T15:02:06.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:02:07.002 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:06.984+0000 7ff6a5c8b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T15:02:07.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:07.028 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:02:07.028 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T15:02:07.037 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:06.746+0000 7f22dc7f5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T15:02:07.037 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:02:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z[34730]: 2024-04-25T15:02:06.963+0000 7f22dc7f5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T15:02:07.037 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:06.745+0000 7f5a8bbe0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T15:02:07.037 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:02:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y[33828]: 2024-04-25T15:02:06.962+0000 7f5a8bbe0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T15:02:07.072 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T15:02:07.072 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:02:07.072 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T15:02:07.155 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T15:02:07.155 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T15:02:07.155 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/8662f957-a607-4fca-9c75-81aedaaa8904 2024-04-25T15:02:07.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:07.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/8662f957-a607-4fca-9c75-81aedaaa8904 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T15:02:07.315 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T15:02:07.316 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T15:02:07.433 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:07.119+0000 7ff6a5c8b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: Standby manager daemon y restarted 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: Standby manager daemon y started 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: Standby manager daemon z restarted 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: Standby manager daemon z started 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T15:02:07.434 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:02:07.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T15:02:07.435 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-mon[27862]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:02:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: Standby manager daemon y restarted 2024-04-25T15:02:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: Standby manager daemon y started 2024-04-25T15:02:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T15:02:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: Standby manager daemon z restarted 2024-04-25T15:02:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: Standby manager daemon z started 2024-04-25T15:02:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T15:02:07.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:02:07.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:02:07.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T15:02:07.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:02:07.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T15:02:07.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:07 smithi138 ceph-mon[32875]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:02:07.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: Standby manager daemon y restarted 2024-04-25T15:02:07.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: Standby manager daemon y started 2024-04-25T15:02:07.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T15:02:07.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: Standby manager daemon z restarted 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: Standby manager daemon z started 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.137:0/961027686' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T15:02:07.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:07 smithi137 ceph-mon[32439]: from='mgr.? 172.21.15.138:0/1745261422' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:02:07.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:07.457+0000 7ff6a5c8b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T15:02:07.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:07.643+0000 7ff6a5c8b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T15:02:08.038 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:07.776+0000 7ff6a5c8b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T15:02:08.038 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:07.933+0000 7ff6a5c8b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T15:02:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T15:02:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T15:02:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.137:3300,172.21.15.138:3300" 2024-04-25T15:02:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with new device syntax: 0@f04d929c-0312-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/8662f957-a607-4fca-9c75-81aedaaa8904 2024-04-25T15:02:08.116 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.94:3300/172.21.15.137:3300/172.21.15.138:3300" will pass to kernel 2024-04-25T15:02:08.119 INFO:teuthology.orchestra.run.smithi094.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T15:02:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: You just mounted a file system that supports labels which does not 2024-04-25T15:02:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T15:02:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T15:02:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T15:02:08.122 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T15:02:08.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:02:08.122 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T15:02:08.191 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T15:02:08.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T15:02:08.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:08.312 INFO:teuthology.orchestra.run.smithi094.stdout:client25166 2024-04-25T15:02:08.312 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T15:02:08.312 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T15:02:08.321 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:08 smithi094 ceph-mon[27862]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T15:02:08.359 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T15:02:08.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:08 smithi138 ceph-mon[32875]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T15:02:08.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:08 smithi137 ceph-mon[32439]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-04-25T15:02:08.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T15:02:08.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:08.508 INFO:teuthology.orchestra.run.smithi094.stdout:client25166 2024-04-25T15:02:08.508 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T15:02:08.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T15:02:08.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:08.621 INFO:teuthology.orchestra.run.smithi094.stdout:client25166 2024-04-25T15:02:08.621 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f04d929c-0312-11ef-bc93-c7b262605968.client25166/status') 2024-04-25T15:02:08.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:02:08.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:08 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:08.320+0000 7ff6a5c8b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T15:02:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.25166 (3)172.21.15.94:0/2917707085 2024-04-25T15:02:08.739 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-04-25T15:02:08.739 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-04-25T15:02:08.739 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-04-25T15:02:08.739 INFO:teuthology.orchestra.run.smithi094.stderr:71 bytes copied, 7.4074e-05 s, 959 kB/s 2024-04-25T15:02:08.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:02:08.751 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T15:02:08.782 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T15:02:08.816 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T15:02:08.816 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:02:08.817 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 -- 172.21.15.94:0/3529136679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 msgr2=0x7f66a8103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 --2- 172.21.15.94:0/3529136679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 0x7f66a8103440 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6694009900 tx=0x7f669402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 -- 172.21.15.94:0/3529136679 shutdown_connections 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 --2- 172.21.15.94:0/3529136679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a8103980 0x7f66a810a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 --2- 172.21.15.94:0/3529136679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 0x7f66a8103440 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 --2- 172.21.15.94:0/3529136679 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66a8101e70 0x7f66a8102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 -- 172.21.15.94:0/3529136679 >> 172.21.15.94:0/3529136679 conn(0x7f66a80fd3e0 msgr2=0x7f66a80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 -- 172.21.15.94:0/3529136679 shutdown_connections 2024-04-25T15:02:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.008+0000 7f66ae200640 1 -- 172.21.15.94:0/3529136679 wait complete. 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.009+0000 7f66ae200640 1 Processor -- start 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.009+0000 7f66ae200640 1 -- start start 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ae200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66a8101e70 0x7f66a819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ae200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 0x7f66a819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ae200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a8103980 0x7f66a81a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ae200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810d960 con 0x7f66a8102fc0 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ae200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810d7e0 con 0x7f66a8101e70 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ae200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810dae0 con 0x7f66a8103980 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 0x7f66a819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66a8101e70 0x7f66a819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66a8101e70 0x7f66a819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38584/0 (socket says 172.21.15.94:38584) 2024-04-25T15:02:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66a7e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 0x7f66a819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51864/0 (socket says 172.21.15.94:51864) 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 -- 172.21.15.94:0/154671911 learned_addr learned my addr 172.21.15.94:0/154671911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ad800640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a8103980 0x7f66a81a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 -- 172.21.15.94:0/154671911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a8103980 msgr2=0x7f66a81a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66a8103980 0x7f66a81a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 -- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 msgr2=0x7f66a819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66a8102fc0 0x7f66a819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.010+0000 7f66ace00640 1 -- 172.21.15.94:0/154671911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a81a1e40 con 0x7f66a8101e70 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.011+0000 7f66ace00640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66a8101e70 0x7f66a819a220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f669c00e9f0 tx=0x7f669c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.011+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c00cd80 con 0x7f66a8101e70 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.011+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a81a2130 con 0x7f66a8101e70 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.011+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669c0108e0 con 0x7f66a8101e70 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.011+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c018980 con 0x7f66a8101e70 2024-04-25T15:02:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.011+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a81a26a0 con 0x7f66a8101e70 2024-04-25T15:02:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.012+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a81a2900 con 0x7f66a8101e70 2024-04-25T15:02:09.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.014+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f669c010430 con 0x7f66a8101e70 2024-04-25T15:02:09.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.016+0000 7f66a5600640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66900a3c70 0x7f66900a6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:09.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.016+0000 7f66a7e00640 1 -- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66900a3c70 msgr2=0x7f66900a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:09.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.016+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8240+0+0 (secure 0 0 0) 0x7f669c0d0be0 con 0x7f66a8101e70 2024-04-25T15:02:09.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.016+0000 7f66a7e00640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66900a3c70 0x7f66900a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:02:09.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.018+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f669c09cf30 con 0x7f66a8101e70 2024-04-25T15:02:09.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.217+0000 7f66a7e00640 1 -- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66900a3c70 msgr2=0x7f66900a6130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2485357518 2024-04-25T15:02:09.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.217+0000 7f66a7e00640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66900a3c70 0x7f66900a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:02:09.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.245+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 --> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f66a80758a0 con 0x7f66900a3c70 2024-04-25T15:02:09.412 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.118+0000 7ff6a5c8b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T15:02:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.436+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f669c0995a0 con 0x7f66a8101e70 2024-04-25T15:02:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.437+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66900a3c70 msgr2=0x7f66900a6130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:02:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:09.437+0000 7f66a5600640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2485357518,v1:172.21.15.94:6801/2485357518] conn(0x7f66900a3c70 0x7f66900a6130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:09.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.410+0000 7ff6a5c8b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T15:02:09.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: Active manager daemon x restarted 2024-04-25T15:02:09.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: Activating manager daemon x 2024-04-25T15:02:09.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T15:02:09.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: mgrmap e21: x(active, starting, since 0.0252539s), standbys: z, y 2024-04-25T15:02:09.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T15:02:09.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T15:02:09.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:02:09.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:02:09.715 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:02:09.716 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:02:09.716 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:02:09.716 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-mon[27862]: Manager daemon x is now available 2024-04-25T15:02:09.724 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: Active manager daemon x restarted 2024-04-25T15:02:09.724 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: Activating manager daemon x 2024-04-25T15:02:09.724 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T15:02:09.724 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: mgrmap e21: x(active, starting, since 0.0252539s), standbys: z, y 2024-04-25T15:02:09.724 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:02:09.725 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T15:02:09.726 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T15:02:09.727 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:02:09.727 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:02:09.727 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:02:09.727 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:02:09.727 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:02:09.727 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:02:09.727 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:09 smithi138 ceph-mon[32875]: Manager daemon x is now available 2024-04-25T15:02:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: Active manager daemon x restarted 2024-04-25T15:02:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: Activating manager daemon x 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: mgrmap e21: x(active, starting, since 0.0252539s), standbys: z, y 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T15:02:09.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:02:09.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:02:09.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T15:02:09.735 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:02:09.736 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:09 smithi137 ceph-mon[32439]: Manager daemon x is now available 2024-04-25T15:02:10.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5fc000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5fc000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5fc000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5fc000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5fc000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5f8400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5f8400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5f8400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5f8400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:02:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:02:09.731+0000 7ff5f8400700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:02:10.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.447+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mon.2 v2:172.21.15.137:3300/0 8 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f669c0ce720 con 0x7f66a8101e70 2024-04-25T15:02:10.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.448+0000 7f66a5600640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f6690052820 0x7f6690052c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:10.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.448+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f669c09dfa0 con 0x7f6690052820 2024-04-25T15:02:10.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.454+0000 7f66a7e00640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f6690052820 0x7f6690052c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:10.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.463+0000 7f66a7e00640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f6690052820 0x7f6690052c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6694009230 tx=0x7f66940314a0 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:10.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.554+0000 7f66a5600640 1 -- 172.21.15.94:0/154671911 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f669c09dfa0 con 0x7f6690052820 2024-04-25T15:02:10.556 INFO:teuthology.orchestra.run.smithi094.stdout:Schedule set for path / 2024-04-25T15:02:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f6690052820 msgr2=0x7f6690052c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:10.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f6690052820 0x7f6690052c00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6694009230 tx=0x7f66940314a0 comp rx=0 tx=0).stop 2024-04-25T15:02:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66a8101e70 msgr2=0x7f66a819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 --2- 172.21.15.94:0/154671911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f66a8101e70 0x7f66a819a220 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f669c00e9f0 tx=0x7f669c00eeb0 comp rx=0 tx=0).stop 2024-04-25T15:02:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ad800640 1 -- 172.21.15.94:0/154671911 reap_dead start 2024-04-25T15:02:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 shutdown_connections 2024-04-25T15:02:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 >> 172.21.15.94:0/154671911 conn(0x7f66a80fd3e0 msgr2=0x7f66a8074180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 shutdown_connections 2024-04-25T15:02:10.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.558+0000 7f66ae200640 1 -- 172.21.15.94:0/154671911 wait complete. 2024-04-25T15:02:10.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T15:02:10.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:02:10.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:02:10.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:02:10.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:02:10.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:02:10.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: [25/Apr/2024:15:02:09] ENGINE Bus STARTING 2024-04-25T15:02:10.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: [25/Apr/2024:15:02:10] ENGINE Serving on https://172.21.15.94:7150 2024-04-25T15:02:10.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: [25/Apr/2024:15:02:10] ENGINE Serving on http://172.21.15.94:8765 2024-04-25T15:02:10.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: [25/Apr/2024:15:02:10] ENGINE Bus STARTED 2024-04-25T15:02:10.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:10 smithi137 ceph-mon[32439]: mgrmap e22: x(active, since 1.043s), standbys: z, y 2024-04-25T15:02:10.758 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:02:10.758 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:02:10.758 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:02:10.758 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:02:10.758 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:02:10.758 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: [25/Apr/2024:15:02:09] ENGINE Bus STARTING 2024-04-25T15:02:10.759 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: [25/Apr/2024:15:02:10] ENGINE Serving on https://172.21.15.94:7150 2024-04-25T15:02:10.759 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: [25/Apr/2024:15:02:10] ENGINE Serving on http://172.21.15.94:8765 2024-04-25T15:02:10.759 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: [25/Apr/2024:15:02:10] ENGINE Bus STARTED 2024-04-25T15:02:10.759 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:10 smithi138 ceph-mon[32875]: mgrmap e22: x(active, since 1.043s), standbys: z, y 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 -- 172.21.15.94:0/600199422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 msgr2=0x7f18581045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 --2- 172.21.15.94:0/600199422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 0x7f18581045f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1848009900 tx=0x7f184802a410 comp rx=0 tx=0).stop 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 -- 172.21.15.94:0/600199422 shutdown_connections 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 --2- 172.21.15.94:0/600199422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1858105d80 0x7f185810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 --2- 172.21.15.94:0/600199422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f18581053c0 0x7f1858105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 --2- 172.21.15.94:0/600199422 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 0x7f18581045f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 -- 172.21.15.94:0/600199422 >> 172.21.15.94:0/600199422 conn(0x7f18580ff740 msgr2=0x7f1858101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 -- 172.21.15.94:0/600199422 shutdown_connections 2024-04-25T15:02:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.758+0000 7f185d800640 1 -- 172.21.15.94:0/600199422 wait complete. 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.759+0000 7f185d800640 1 Processor -- start 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.759+0000 7f185d800640 1 -- start start 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f185d800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 0x7f185819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f185d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18581053c0 0x7f185819a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f185d800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1858105d80 0x7f18581a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f185d800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f185810fd60 con 0x7f1858105d80 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f185d800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f185810fbe0 con 0x7f18581041d0 2024-04-25T15:02:10.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f185d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f185810fee0 con 0x7f18581053c0 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f1856000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18581053c0 0x7f185819a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f1856000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18581053c0 0x7f185819a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38136/0 (socket says 172.21.15.94:38136) 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f1856000640 1 -- 172.21.15.94:0/4001956261 learned_addr learned my addr 172.21.15.94:0/4001956261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f1856a00640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 0x7f185819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.760+0000 7f1857400640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1858105d80 0x7f18581a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f1856000640 1 -- 172.21.15.94:0/4001956261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 msgr2=0x7f185819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f1856000640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 0x7f185819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f1856000640 1 -- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1858105d80 msgr2=0x7f18581a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f1856000640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1858105d80 0x7f18581a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f1856000640 1 -- 172.21.15.94:0/4001956261 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18581a1f80 con 0x7f18581053c0 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f1856a00640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 0x7f185819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:10.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f1856000640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18581053c0 0x7f185819a8a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9f0 tx=0x7f184400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:10.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f184f400640 1 -- 172.21.15.94:0/4001956261 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184400cd80 con 0x7f18581053c0 2024-04-25T15:02:10.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f184f400640 1 -- 172.21.15.94:0/4001956261 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18440108e0 con 0x7f18581053c0 2024-04-25T15:02:10.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f184f400640 1 -- 172.21.15.94:0/4001956261 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844018980 con 0x7f18581053c0 2024-04-25T15:02:10.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18581a2270 con 0x7f18581053c0 2024-04-25T15:02:10.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.761+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a2810 con 0x7f18581053c0 2024-04-25T15:02:10.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.762+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185810b4f0 con 0x7f18581053c0 2024-04-25T15:02:10.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.765+0000 7f184f400640 1 -- 172.21.15.94:0/4001956261 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1844010430 con 0x7f18581053c0 2024-04-25T15:02:10.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.766+0000 7f184f400640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f183c0a3ae0 0x7f183c0a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:10.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.766+0000 7f184f400640 1 -- 172.21.15.94:0/4001956261 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f18440d04c0 con 0x7f18581053c0 2024-04-25T15:02:10.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.766+0000 7f1856a00640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f183c0a3ae0 0x7f183c0a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:10.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.767+0000 7f1856a00640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f183c0a3ae0 0x7f183c0a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1848006010 tx=0x7f18480314a0 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:10.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.770+0000 7f184f400640 1 -- 172.21.15.94:0/4001956261 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f184409d040 con 0x7f18581053c0 2024-04-25T15:02:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: [25/Apr/2024:15:02:09] ENGINE Bus STARTING 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: [25/Apr/2024:15:02:10] ENGINE Serving on https://172.21.15.94:7150 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: [25/Apr/2024:15:02:10] ENGINE Serving on http://172.21.15.94:8765 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: [25/Apr/2024:15:02:10] ENGINE Bus STARTED 2024-04-25T15:02:10.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:10 smithi094 ceph-mon[27862]: mgrmap e22: x(active, since 1.043s), standbys: z, y 2024-04-25T15:02:10.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:10.989+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f18580624f0 con 0x7f183c0a3ae0 2024-04-25T15:02:11.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.100+0000 7f184f400640 1 -- 172.21.15.94:0/4001956261 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f18580624f0 con 0x7f183c0a3ae0 2024-04-25T15:02:11.102 INFO:teuthology.orchestra.run.smithi094.stdout:Retention added to path / 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f183c0a3ae0 msgr2=0x7f183c0a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f183c0a3ae0 0x7f183c0a5fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1848006010 tx=0x7f18480314a0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18581053c0 msgr2=0x7f185819a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18581053c0 0x7f185819a8a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f184400e9f0 tx=0x7f184400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 shutdown_connections 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f183c0a3ae0 0x7f183c0a5fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1858105d80 0x7f18581a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18581053c0 0x7f185819a8a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 --2- 172.21.15.94:0/4001956261 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f18581041d0 0x7f185819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.103+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 >> 172.21.15.94:0/4001956261 conn(0x7f18580ff740 msgr2=0x7f18581086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.104+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 shutdown_connections 2024-04-25T15:02:11.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.104+0000 7f185d800640 1 -- 172.21.15.94:0/4001956261 wait complete. 2024-04-25T15:02:11.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T15:02:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 -- 172.21.15.94:0/4120772463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c0749b0 msgr2=0x7fcc1c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 --2- 172.21.15.94:0/4120772463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c0749b0 0x7fcc1c074e30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009860 tx=0x7fcc0802a340 comp rx=0 tx=0).stop 2024-04-25T15:02:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 -- 172.21.15.94:0/4120772463 shutdown_connections 2024-04-25T15:02:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 --2- 172.21.15.94:0/4120772463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc1c109d30 0x7fcc1c10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 --2- 172.21.15.94:0/4120772463 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c0749b0 0x7fcc1c074e30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 --2- 172.21.15.94:0/4120772463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc1c075f90 0x7fcc1c0743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 -- 172.21.15.94:0/4120772463 >> 172.21.15.94:0/4120772463 conn(0x7fcc1c0fd3e0 msgr2=0x7fcc1c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 -- 172.21.15.94:0/4120772463 shutdown_connections 2024-04-25T15:02:11.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.291+0000 7fcc22800640 1 -- 172.21.15.94:0/4120772463 wait complete. 2024-04-25T15:02:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 Processor -- start 2024-04-25T15:02:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 -- start start 2024-04-25T15:02:11.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc1c0749b0 0x7fcc1c19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c075f90 0x7fcc1c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc1c109d30 0x7fcc1c1a17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c10fb30 con 0x7fcc1c0749b0 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c10f9b0 con 0x7fcc1c075f90 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.292+0000 7fcc22800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc1c10fcb0 con 0x7fcc1c109d30 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc21400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc1c0749b0 0x7fcc1c19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc20a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c075f90 0x7fcc1c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc1c109d30 0x7fcc1c1a17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc1c109d30 0x7fcc1c1a17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38156/0 (socket says 172.21.15.94:38156) 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc21e00640 1 -- 172.21.15.94:0/1186899323 learned_addr learned my addr 172.21.15.94:0/1186899323 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc20a00640 1 -- 172.21.15.94:0/1186899323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc1c109d30 msgr2=0x7fcc1c1a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc20a00640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc1c109d30 0x7fcc1c1a17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc20a00640 1 -- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc1c0749b0 msgr2=0x7fcc1c19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc20a00640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc1c0749b0 0x7fcc1c19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc20a00640 1 -- 172.21.15.94:0/1186899323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc1c1a1e20 con 0x7fcc1c075f90 2024-04-25T15:02:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc21400640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc1c0749b0 0x7fcc1c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.293+0000 7fcc20a00640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c075f90 0x7fcc1c19a740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc0802a860 tx=0x7fcc08032dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:11.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.294+0000 7fcc1a000640 1 -- 172.21.15.94:0/1186899323 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08040020 con 0x7fcc1c075f90 2024-04-25T15:02:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.294+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc1c1a20b0 con 0x7fcc1c075f90 2024-04-25T15:02:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.294+0000 7fcc1a000640 1 -- 172.21.15.94:0/1186899323 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc08030dd0 con 0x7fcc1c075f90 2024-04-25T15:02:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.294+0000 7fcc1a000640 1 -- 172.21.15.94:0/1186899323 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08037040 con 0x7fcc1c075f90 2024-04-25T15:02:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.294+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc1c1a25f0 con 0x7fcc1c075f90 2024-04-25T15:02:11.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.295+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c108a60 con 0x7fcc1c075f90 2024-04-25T15:02:11.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.297+0000 7fcc1a000640 1 -- 172.21.15.94:0/1186899323 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fcc08043020 con 0x7fcc1c075f90 2024-04-25T15:02:11.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.298+0000 7fcc1a000640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcc040a3990 0x7fcc040a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.298+0000 7fcc1a000640 1 -- 172.21.15.94:0/1186899323 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcc080eff50 con 0x7fcc1c075f90 2024-04-25T15:02:11.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.298+0000 7fcc21400640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcc040a3990 0x7fcc040a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.299+0000 7fcc21400640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcc040a3990 0x7fcc040a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005fd0 tx=0x7fcc10010040 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:11.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.302+0000 7fcc1a000640 1 -- 172.21.15.94:0/1186899323 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc0804a020 con 0x7fcc1c075f90 2024-04-25T15:02:11.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.535+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fcc1c062390 con 0x7fcc040a3990 2024-04-25T15:02:11.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.555+0000 7fcc1a000640 1 -- 172.21.15.94:0/1186899323 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fcc1c062390 con 0x7fcc040a3990 2024-04-25T15:02:11.557 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T15:02:10", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T15:02:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcc040a3990 msgr2=0x7fcc040a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcc040a3990 0x7fcc040a5e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcc10005fd0 tx=0x7fcc10010040 comp rx=0 tx=0).stop 2024-04-25T15:02:11.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c075f90 msgr2=0x7fcc1c19a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c075f90 0x7fcc1c19a740 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc0802a860 tx=0x7fcc08032dc0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 shutdown_connections 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcc040a3990 0x7fcc040a5e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc1c109d30 0x7fcc1c1a17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcc1c075f90 0x7fcc1c19a740 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 --2- 172.21.15.94:0/1186899323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcc1c0749b0 0x7fcc1c19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 >> 172.21.15.94:0/1186899323 conn(0x7fcc1c0fd3e0 msgr2=0x7fcc1c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 shutdown_connections 2024-04-25T15:02:11.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.558+0000 7fcc22800640 1 -- 172.21.15.94:0/1186899323 wait complete. 2024-04-25T15:02:11.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T15:02:11.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 -- 172.21.15.94:0/1812541567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 msgr2=0x7f69f4105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 --2- 172.21.15.94:0/1812541567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 0x7f69f4105840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009900 tx=0x7f69e002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 -- 172.21.15.94:0/1812541567 shutdown_connections 2024-04-25T15:02:11.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 --2- 172.21.15.94:0/1812541567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f4105d80 0x7f69f410c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 --2- 172.21.15.94:0/1812541567 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 0x7f69f4105840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 --2- 172.21.15.94:0/1812541567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f41041d0 0x7f69f41045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 -- 172.21.15.94:0/1812541567 >> 172.21.15.94:0/1812541567 conn(0x7f69f40ff740 msgr2=0x7f69f4101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:11.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 -- 172.21.15.94:0/1812541567 shutdown_connections 2024-04-25T15:02:11.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.769+0000 7f69fb800640 1 -- 172.21.15.94:0/1812541567 wait complete. 2024-04-25T15:02:11.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.770+0000 7f69fb800640 1 Processor -- start 2024-04-25T15:02:11.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.770+0000 7f69fb800640 1 -- start start 2024-04-25T15:02:11.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.770+0000 7f69fb800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f41041d0 0x7f69f419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69fb800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 0x7f69f419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69fb800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4105d80 0x7f69f41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69fb800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f69f410fd60 con 0x7f69f4105d80 2024-04-25T15:02:11.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69fb800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f69f410fbe0 con 0x7f69f41053c0 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69fb800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69f410fee0 con 0x7f69f41041d0 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f41041d0 0x7f69f419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f9800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4105d80 0x7f69f41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 0x7f69f419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f41041d0 0x7f69f419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:38198/0 (socket says 172.21.15.94:38198) 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 -- 172.21.15.94:0/3131266183 learned_addr learned my addr 172.21.15.94:0/3131266183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 -- 172.21.15.94:0/3131266183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 msgr2=0x7f69f419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 0x7f69f419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 -- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4105d80 msgr2=0x7f69f41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4105d80 0x7f69f41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.771+0000 7f69f8e00640 1 -- 172.21.15.94:0/3131266183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f41a1f80 con 0x7f69f41041d0 2024-04-25T15:02:11.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.772+0000 7f69f8e00640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f41041d0 0x7f69f419a360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e9c0 tx=0x7f69e800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.772+0000 7f69f1600640 1 -- 172.21.15.94:0/3131266183 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e800cd80 con 0x7f69f41041d0 2024-04-25T15:02:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.772+0000 7f69f1600640 1 -- 172.21.15.94:0/3131266183 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69e80035f0 con 0x7f69f41041d0 2024-04-25T15:02:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.772+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f41a2270 con 0x7f69f41041d0 2024-04-25T15:02:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.772+0000 7f69f1600640 1 -- 172.21.15.94:0/3131266183 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8010690 con 0x7f69f41041d0 2024-04-25T15:02:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.772+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f41a27e0 con 0x7f69f41041d0 2024-04-25T15:02:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.773+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f410b4f0 con 0x7f69f41041d0 2024-04-25T15:02:11.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.775+0000 7f69f1600640 1 -- 172.21.15.94:0/3131266183 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f69e80037a0 con 0x7f69f41041d0 2024-04-25T15:02:11.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.776+0000 7f69f1600640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f69dc0a3e40 0x7f69dc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.776+0000 7f69f1600640 1 -- 172.21.15.94:0/3131266183 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f69e80d0b30 con 0x7f69f41041d0 2024-04-25T15:02:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.776+0000 7f69f3e00640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f69dc0a3e40 0x7f69dc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:11.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.777+0000 7f69f3e00640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f69dc0a3e40 0x7f69dc0a6300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69e0031500 tx=0x7f69e0005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:11.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:11.780+0000 7f69f1600640 1 -- 172.21.15.94:0/3131266183 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69e809d180 con 0x7f69f41041d0 2024-04-25T15:02:11.942 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.943 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:11 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:11.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:11.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:11 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:12.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.010+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f69f4062390 con 0x7f69dc0a3e40 2024-04-25T15:02:12.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.020+0000 7f69f1600640 1 -- 172.21.15.94:0/3131266183 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f69f4062390 con 0x7f69dc0a3e40 2024-04-25T15:02:12.022 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-04-25T15:02:12.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f69dc0a3e40 msgr2=0x7f69dc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:12.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f69dc0a3e40 0x7f69dc0a6300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69e0031500 tx=0x7f69e0005a20 comp rx=0 tx=0).stop 2024-04-25T15:02:12.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f41041d0 msgr2=0x7f69f419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:12.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f41041d0 0x7f69f419a360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f69e800e9c0 tx=0x7f69e800ee80 comp rx=0 tx=0).stop 2024-04-25T15:02:12.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 shutdown_connections 2024-04-25T15:02:12.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f69dc0a3e40 0x7f69dc0a6300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69f4105d80 0x7f69f41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f69f41053c0 0x7f69f419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 --2- 172.21.15.94:0/3131266183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69f41041d0 0x7f69f419a360 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.024+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 >> 172.21.15.94:0/3131266183 conn(0x7f69f40ff740 msgr2=0x7f69f41086c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.025+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 shutdown_connections 2024-04-25T15:02:12.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:12.025+0000 7f69fb800640 1 -- 172.21.15.94:0/3131266183 wait complete. 2024-04-25T15:02:12.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T15:02:12.131 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T15:02:12.150 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T15:02:12.192 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:02:12.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T15:02:12.196 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T15:02:12.226 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T15:02:12.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:02:12.227 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='client.25172 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:12.227 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:12.228 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:12.228 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:12.228 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='client.25198 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:12.228 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:12.228 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:12.228 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:12.228 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:11 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:12.763 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:13.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:13.069 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.070 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:12 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:12 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:13.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='client.25214 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:13.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-25T15:02:13.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='client.25216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:02:13.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:13.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:12 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T15:02:14.062 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:13 smithi094 ceph-mon[27862]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:02:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T15:02:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:02:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:14.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T15:02:14.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:13 smithi138 ceph-mon[32875]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:02:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T15:02:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:02:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:14.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T15:02:14.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:13 smithi137 ceph-mon[32439]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 -- 172.21.15.94:0/3233047242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 msgr2=0x7f38041098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 --2- 172.21.15.94:0/3233047242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 0x7f38041098e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009990 tx=0x7f37f0009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 -- 172.21.15.94:0/3233047242 shutdown_connections 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 --2- 172.21.15.94:0/3233047242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3804109fc0 0x7f3804111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 --2- 172.21.15.94:0/3233047242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3804104ee0 0x7f38041052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 --2- 172.21.15.94:0/3233047242 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 0x7f38041098e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 -- 172.21.15.94:0/3233047242 >> 172.21.15.94:0/3233047242 conn(0x7f3804100770 msgr2=0x7f3804102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 -- 172.21.15.94:0/3233047242 shutdown_connections 2024-04-25T15:02:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.883+0000 7f380c600700 1 -- 172.21.15.94:0/3233047242 wait complete. 2024-04-25T15:02:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.884+0000 7f380c600700 1 Processor -- start 2024-04-25T15:02:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.885+0000 7f380c600700 1 -- start start 2024-04-25T15:02:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.885+0000 7f380c600700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3804104ee0 0x7f380419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:14.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.885+0000 7f380c600700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 0x7f380419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.885+0000 7f380c600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3804109fc0 0x7f380419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.885+0000 7f380c600700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3804114e50 con 0x7f3804109fc0 2024-04-25T15:02:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.885+0000 7f380c600700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3804114cd0 con 0x7f3804105890 2024-04-25T15:02:14.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.885+0000 7f380c600700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3804114fd0 con 0x7f3804104ee0 2024-04-25T15:02:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f380a600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3804109fc0 0x7f380419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 0x7f380419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809c00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3804104ee0 0x7f380419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 0x7f380419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:38778/0 (socket says 172.21.15.94:38778) 2024-04-25T15:02:14.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809200700 1 -- 172.21.15.94:0/3118066627 learned_addr learned my addr 172.21.15.94:0/3118066627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f380a600700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3804109fc0 0x7f380419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52090/0 (socket says 172.21.15.94:52090) 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809c00700 1 -- 172.21.15.94:0/3118066627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 msgr2=0x7f380419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809c00700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 0x7f380419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809c00700 1 -- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3804109fc0 msgr2=0x7f380419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809c00700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3804109fc0 0x7f380419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809c00700 1 -- 172.21.15.94:0/3118066627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38041a0320 con 0x7f3804104ee0 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.886+0000 7f3809c00700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3804104ee0 0x7f380419b360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37f800efc0 tx=0x7f37f800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.887+0000 7f3802000700 1 -- 172.21.15.94:0/3118066627 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f80092e0 con 0x7f3804104ee0 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.887+0000 7f3802000700 1 -- 172.21.15.94:0/3118066627 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37f8004bb0 con 0x7f3804104ee0 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.887+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38041a05b0 con 0x7f3804104ee0 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.887+0000 7f3802000700 1 -- 172.21.15.94:0/3118066627 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8004040 con 0x7f3804104ee0 2024-04-25T15:02:14.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.887+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38041a7e90 con 0x7f3804104ee0 2024-04-25T15:02:14.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.888+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38041063c0 con 0x7f3804104ee0 2024-04-25T15:02:14.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.890+0000 7f3802000700 1 -- 172.21.15.94:0/3118066627 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f37f801b030 con 0x7f3804104ee0 2024-04-25T15:02:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.891+0000 7f3802000700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f37ec0a3e40 0x7f37ec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.892+0000 7f3802000700 1 -- 172.21.15.94:0/3118066627 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f37f80d0520 con 0x7f3804104ee0 2024-04-25T15:02:14.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.892+0000 7f3809200700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f37ec0a3e40 0x7f37ec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:14.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.893+0000 7f3809200700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f37ec0a3e40 0x7f37ec0a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37f000b790 tx=0x7f37f0005960 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:14.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:14.896+0000 7f3802000700 1 -- 172.21.15.94:0/3118066627 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37f8022020 con 0x7f3804104ee0 2024-04-25T15:02:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:14 smithi094 ceph-mon[27862]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:14 smithi094 ceph-mon[27862]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T15:02:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:14 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:15.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.129+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3804062e80 con 0x7f37ec0a3e40 2024-04-25T15:02:15.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.144+0000 7f3802000700 1 -- 172.21.15.94:0/3118066627 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257703 (secure 0 0 0) 0x7f37f0066ec0 con 0x7f37ec0a3e40 2024-04-25T15:02:15.151 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:15.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f37ec0a3e40 msgr2=0x7f37ec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:15.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f37ec0a3e40 0x7f37ec0a6300 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37f000b790 tx=0x7f37f0005960 comp rx=0 tx=0).stop 2024-04-25T15:02:15.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3804104ee0 msgr2=0x7f380419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:15.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3804104ee0 0x7f380419b360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37f800efc0 tx=0x7f37f800c3a0 comp rx=0 tx=0).stop 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 shutdown_connections 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3804109fc0 0x7f380419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3804104ee0 0x7f380419b360 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f37ec0a3e40 0x7f37ec0a6300 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 --2- 172.21.15.94:0/3118066627 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3804105890 0x7f380419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 >> 172.21.15.94:0/3118066627 conn(0x7f3804100770 msgr2=0x7f3804102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 shutdown_connections 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:15.149+0000 7f380c600700 1 -- 172.21.15.94:0/3118066627 wait complete. 2024-04-25T15:02:15.154 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T15:02:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:14 smithi138 ceph-mon[32875]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:14 smithi138 ceph-mon[32875]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T15:02:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:14 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:15.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:14 smithi137 ceph-mon[32439]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:15.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:14 smithi137 ceph-mon[32439]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-25T15:02:15.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:14 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:15.980 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T15:02:13.449821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1183976,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":526,"num_read_kb":416,"num_write":377,"num_write_kb":4342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380416,"kb_used_data":14596,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561408,"statfs":{"total":1151940427776,"available":1151550881792,"internally_reserved":0,"allocated":14946304,"data_stored":8803205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.980460"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.819097+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:02:09.819097+0000","last_peered":"2024-04-25T15:02:09.819097+0000","last_clean":"2024-04-25T15:02:09.819097+0000","last_became_active":"2024-04-25T14:59:46.311856+0000","last_became_peered":"2024-04-25T14:59:46.311856+0000","last_unstale":"2024-04-25T15:02:09.819097+0000","last_undegraded":"2024-04-25T15:02:09.819097+0000","last_fullsized":"2024-04-25T15:02:09.819097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:07.469645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427851+0000","last_change":"2024-04-25T14:59:46.310292+0000","last_active":"2024-04-25T15:02:09.427851+0000","last_peered":"2024-04-25T15:02:09.427851+0000","last_clean":"2024-04-25T15:02:09.427851+0000","last_became_active":"2024-04-25T14:59:46.309812+0000","last_became_peered":"2024-04-25T14:59:46.309812+0000","last_unstale":"2024-04-25T15:02:09.427851+0000","last_undegraded":"2024-04-25T15:02:09.427851+0000","last_fullsized":"2024-04-25T15:02:09.427851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:52.448406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428156+0000","last_change":"2024-04-25T14:59:46.300987+0000","last_active":"2024-04-25T15:02:09.428156+0000","last_peered":"2024-04-25T15:02:09.428156+0000","last_clean":"2024-04-25T15:02:09.428156+0000","last_became_active":"2024-04-25T14:59:46.299866+0000","last_became_peered":"2024-04-25T14:59:46.299866+0000","last_unstale":"2024-04-25T15:02:09.428156+0000","last_undegraded":"2024-04-25T15:02:09.428156+0000","last_fullsized":"2024-04-25T15:02:09.428156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:09.831131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457562+0000","last_change":"2024-04-25T14:59:46.304278+0000","last_active":"2024-04-25T15:02:09.457562+0000","last_peered":"2024-04-25T15:02:09.457562+0000","last_clean":"2024-04-25T15:02:09.457562+0000","last_became_active":"2024-04-25T14:59:46.303956+0000","last_became_peered":"2024-04-25T14:59:46.303956+0000","last_unstale":"2024-04-25T15:02:09.457562+0000","last_undegraded":"2024-04-25T15:02:09.457562+0000","last_fullsized":"2024-04-25T15:02:09.457562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.752548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562299+0000","last_change":"2024-04-25T14:59:46.323730+0000","last_active":"2024-04-25T15:02:09.562299+0000","last_peered":"2024-04-25T15:02:09.562299+0000","last_clean":"2024-04-25T15:02:09.562299+0000","last_became_active":"2024-04-25T14:59:46.323383+0000","last_became_peered":"2024-04-25T14:59:46.323383+0000","last_unstale":"2024-04-25T15:02:09.562299+0000","last_undegraded":"2024-04-25T15:02:09.562299+0000","last_fullsized":"2024-04-25T15:02:09.562299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.846392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574136+0000","last_change":"2024-04-25T14:59:46.317623+0000","last_active":"2024-04-25T15:02:09.574136+0000","last_peered":"2024-04-25T15:02:09.574136+0000","last_clean":"2024-04-25T15:02:09.574136+0000","last_became_active":"2024-04-25T14:59:46.317255+0000","last_became_peered":"2024-04-25T14:59:46.317255+0000","last_unstale":"2024-04-25T15:02:09.574136+0000","last_undegraded":"2024-04-25T15:02:09.574136+0000","last_fullsized":"2024-04-25T15:02:09.574136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:24.449615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426641+0000","last_change":"2024-04-25T15:00:02.723289+0000","last_active":"2024-04-25T15:02:09.426641+0000","last_peered":"2024-04-25T15:02:09.426641+0000","last_clean":"2024-04-25T15:02:09.426641+0000","last_became_active":"2024-04-25T15:00:02.722968+0000","last_became_peered":"2024-04-25T15:00:02.722968+0000","last_unstale":"2024-04-25T15:02:09.426641+0000","last_undegraded":"2024-04-25T15:02:09.426641+0000","last_fullsized":"2024-04-25T15:02:09.426641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:45.674905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562263+0000","last_change":"2024-04-25T14:59:46.327687+0000","last_active":"2024-04-25T15:02:09.562263+0000","last_peered":"2024-04-25T15:02:09.562263+0000","last_clean":"2024-04-25T15:02:09.562263+0000","last_became_active":"2024-04-25T14:59:46.325905+0000","last_became_peered":"2024-04-25T14:59:46.325905+0000","last_unstale":"2024-04-25T15:02:09.562263+0000","last_undegraded":"2024-04-25T15:02:09.562263+0000","last_fullsized":"2024-04-25T15:02:09.562263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:50.058801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740049+0000","last_change":"2024-04-25T14:59:46.304722+0000","last_active":"2024-04-25T15:02:09.740049+0000","last_peered":"2024-04-25T15:02:09.740049+0000","last_clean":"2024-04-25T15:02:09.740049+0000","last_became_active":"2024-04-25T14:59:46.304391+0000","last_became_peered":"2024-04-25T14:59:46.304391+0000","last_unstale":"2024-04-25T15:02:09.740049+0000","last_undegraded":"2024-04-25T15:02:09.740049+0000","last_fullsized":"2024-04-25T15:02:09.740049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:15.735670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574410+0000","last_change":"2024-04-25T14:59:46.307796+0000","last_active":"2024-04-25T15:02:09.574410+0000","last_peered":"2024-04-25T15:02:09.574410+0000","last_clean":"2024-04-25T15:02:09.574410+0000","last_became_active":"2024-04-25T14:59:46.306741+0000","last_became_peered":"2024-04-25T14:59:46.306741+0000","last_unstale":"2024-04-25T15:02:09.574410+0000","last_undegraded":"2024-04-25T15:02:09.574410+0000","last_fullsized":"2024-04-25T15:02:09.574410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:34.407272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427068+0000","last_change":"2024-04-25T14:59:46.306390+0000","last_active":"2024-04-25T15:02:09.427068+0000","last_peered":"2024-04-25T15:02:09.427068+0000","last_clean":"2024-04-25T15:02:09.427068+0000","last_became_active":"2024-04-25T14:59:46.306026+0000","last_became_peered":"2024-04-25T14:59:46.306026+0000","last_unstale":"2024-04-25T15:02:09.427068+0000","last_undegraded":"2024-04-25T15:02:09.427068+0000","last_fullsized":"2024-04-25T15:02:09.427068+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:59.340382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457480+0000","last_change":"2024-04-25T14:59:46.305840+0000","last_active":"2024-04-25T15:02:09.457480+0000","last_peered":"2024-04-25T15:02:09.457480+0000","last_clean":"2024-04-25T15:02:09.457480+0000","last_became_active":"2024-04-25T14:59:46.304867+0000","last_became_peered":"2024-04-25T14:59:46.304867+0000","last_unstale":"2024-04-25T15:02:09.457480+0000","last_undegraded":"2024-04-25T15:02:09.457480+0000","last_fullsized":"2024-04-25T15:02:09.457480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:58.569011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143698+0000","last_change":"2024-04-25T14:59:46.318044+0000","last_active":"2024-04-25T15:02:10.143698+0000","last_peered":"2024-04-25T15:02:10.143698+0000","last_clean":"2024-04-25T15:02:10.143698+0000","last_became_active":"2024-04-25T14:59:46.315935+0000","last_became_peered":"2024-04-25T14:59:46.315935+0000","last_unstale":"2024-04-25T15:02:10.143698+0000","last_undegraded":"2024-04-25T15:02:10.143698+0000","last_fullsized":"2024-04-25T15:02:10.143698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:42.366802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427570+0000","last_change":"2024-04-25T14:59:46.308565+0000","last_active":"2024-04-25T15:02:09.427570+0000","last_peered":"2024-04-25T15:02:09.427570+0000","last_clean":"2024-04-25T15:02:09.427570+0000","last_became_active":"2024-04-25T14:59:46.306198+0000","last_became_peered":"2024-04-25T14:59:46.306198+0000","last_unstale":"2024-04-25T15:02:09.427570+0000","last_undegraded":"2024-04-25T15:02:09.427570+0000","last_fullsized":"2024-04-25T15:02:09.427570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:12.686162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427475+0000","last_change":"2024-04-25T14:59:46.308313+0000","last_active":"2024-04-25T15:02:09.427475+0000","last_peered":"2024-04-25T15:02:09.427475+0000","last_clean":"2024-04-25T15:02:09.427475+0000","last_became_active":"2024-04-25T14:59:46.305974+0000","last_became_peered":"2024-04-25T14:59:46.305974+0000","last_unstale":"2024-04-25T15:02:09.427475+0000","last_undegraded":"2024-04-25T15:02:09.427475+0000","last_fullsized":"2024-04-25T15:02:09.427475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.841983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.739661+0000","last_change":"2024-04-25T14:59:46.302726+0000","last_active":"2024-04-25T15:02:09.739661+0000","last_peered":"2024-04-25T15:02:09.739661+0000","last_clean":"2024-04-25T15:02:09.739661+0000","last_became_active":"2024-04-25T14:59:46.302406+0000","last_became_peered":"2024-04-25T14:59:46.302406+0000","last_unstale":"2024-04-25T15:02:09.739661+0000","last_undegraded":"2024-04-25T15:02:09.739661+0000","last_fullsized":"2024-04-25T15:02:09.739661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.819221+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:02:09.819221+0000","last_peered":"2024-04-25T15:02:09.819221+0000","last_clean":"2024-04-25T15:02:09.819221+0000","last_became_active":"2024-04-25T14:59:46.311822+0000","last_became_peered":"2024-04-25T14:59:46.311822+0000","last_unstale":"2024-04-25T15:02:09.819221+0000","last_undegraded":"2024-04-25T15:02:09.819221+0000","last_fullsized":"2024-04-25T15:02:09.819221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.128537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559804+0000","last_change":"2024-04-25T14:59:46.313816+0000","last_active":"2024-04-25T15:02:09.559804+0000","last_peered":"2024-04-25T15:02:09.559804+0000","last_clean":"2024-04-25T15:02:09.559804+0000","last_became_active":"2024-04-25T14:59:46.313462+0000","last_became_peered":"2024-04-25T14:59:46.313462+0000","last_unstale":"2024-04-25T15:02:09.559804+0000","last_undegraded":"2024-04-25T15:02:09.559804+0000","last_fullsized":"2024-04-25T15:02:09.559804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:29.402113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426726+0000","last_change":"2024-04-25T14:59:46.317972+0000","last_active":"2024-04-25T15:02:09.426726+0000","last_peered":"2024-04-25T15:02:09.426726+0000","last_clean":"2024-04-25T15:02:09.426726+0000","last_became_active":"2024-04-25T14:59:46.317466+0000","last_became_peered":"2024-04-25T14:59:46.317466+0000","last_unstale":"2024-04-25T15:02:09.426726+0000","last_undegraded":"2024-04-25T15:02:09.426726+0000","last_fullsized":"2024-04-25T15:02:09.426726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:03.706663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559858+0000","last_change":"2024-04-25T14:59:45.320752+0000","last_active":"2024-04-25T15:02:09.559858+0000","last_peered":"2024-04-25T15:02:09.559858+0000","last_clean":"2024-04-25T15:02:09.559858+0000","last_became_active":"2024-04-25T14:59:45.319463+0000","last_became_peered":"2024-04-25T14:59:45.319463+0000","last_unstale":"2024-04-25T15:02:09.559858+0000","last_undegraded":"2024-04-25T15:02:09.559858+0000","last_fullsized":"2024-04-25T15:02:09.559858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:52.608899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563574+0000","last_change":"2024-04-25T14:59:46.323802+0000","last_active":"2024-04-25T15:02:09.563574+0000","last_peered":"2024-04-25T15:02:09.563574+0000","last_clean":"2024-04-25T15:02:09.563574+0000","last_became_active":"2024-04-25T14:59:46.323498+0000","last_became_peered":"2024-04-25T14:59:46.323498+0000","last_unstale":"2024-04-25T15:02:09.563574+0000","last_undegraded":"2024-04-25T15:02:09.563574+0000","last_fullsized":"2024-04-25T15:02:09.563574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:38.236936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563729+0000","last_change":"2024-04-25T14:59:45.335156+0000","last_active":"2024-04-25T15:02:09.563729+0000","last_peered":"2024-04-25T15:02:09.563729+0000","last_clean":"2024-04-25T15:02:09.563729+0000","last_became_active":"2024-04-25T14:59:45.333589+0000","last_became_peered":"2024-04-25T14:59:45.333589+0000","last_unstale":"2024-04-25T15:02:09.563729+0000","last_undegraded":"2024-04-25T15:02:09.563729+0000","last_fullsized":"2024-04-25T15:02:09.563729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428190+0000","last_change":"2024-04-25T14:59:46.319165+0000","last_active":"2024-04-25T15:02:09.428190+0000","last_peered":"2024-04-25T15:02:09.428190+0000","last_clean":"2024-04-25T15:02:09.428190+0000","last_became_active":"2024-04-25T14:59:46.318801+0000","last_became_peered":"2024-04-25T14:59:46.318801+0000","last_unstale":"2024-04-25T15:02:09.428190+0000","last_undegraded":"2024-04-25T15:02:09.428190+0000","last_fullsized":"2024-04-25T15:02:09.428190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:01.806099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562565+0000","last_change":"2024-04-25T14:59:45.317956+0000","last_active":"2024-04-25T15:02:09.562565+0000","last_peered":"2024-04-25T15:02:09.562565+0000","last_clean":"2024-04-25T15:02:09.562565+0000","last_became_active":"2024-04-25T14:59:45.317655+0000","last_became_peered":"2024-04-25T14:59:45.317655+0000","last_unstale":"2024-04-25T15:02:09.562565+0000","last_undegraded":"2024-04-25T15:02:09.562565+0000","last_fullsized":"2024-04-25T15:02:09.562565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:59.008411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740094+0000","last_change":"2024-04-25T14:59:46.311659+0000","last_active":"2024-04-25T15:02:09.740094+0000","last_peered":"2024-04-25T15:02:09.740094+0000","last_clean":"2024-04-25T15:02:09.740094+0000","last_became_active":"2024-04-25T14:59:46.311287+0000","last_became_peered":"2024-04-25T14:59:46.311287+0000","last_unstale":"2024-04-25T15:02:09.740094+0000","last_undegraded":"2024-04-25T15:02:09.740094+0000","last_fullsized":"2024-04-25T15:02:09.740094+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:29.434223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562402+0000","last_change":"2024-04-25T14:59:45.317658+0000","last_active":"2024-04-25T15:02:09.562402+0000","last_peered":"2024-04-25T15:02:09.562402+0000","last_clean":"2024-04-25T15:02:09.562402+0000","last_became_active":"2024-04-25T14:59:45.317311+0000","last_became_peered":"2024-04-25T14:59:45.317311+0000","last_unstale":"2024-04-25T15:02:09.562402+0000","last_undegraded":"2024-04-25T15:02:09.562402+0000","last_fullsized":"2024-04-25T15:02:09.562402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.602412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570739+0000","last_change":"2024-04-25T14:59:46.305456+0000","last_active":"2024-04-25T15:02:09.570739+0000","last_peered":"2024-04-25T15:02:09.570739+0000","last_clean":"2024-04-25T15:02:09.570739+0000","last_became_active":"2024-04-25T14:59:46.305143+0000","last_became_peered":"2024-04-25T14:59:46.305143+0000","last_unstale":"2024-04-25T15:02:09.570739+0000","last_undegraded":"2024-04-25T15:02:09.570739+0000","last_fullsized":"2024-04-25T15:02:09.570739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:03.941200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570545+0000","last_change":"2024-04-25T14:59:45.315407+0000","last_active":"2024-04-25T15:02:09.570545+0000","last_peered":"2024-04-25T15:02:09.570545+0000","last_clean":"2024-04-25T15:02:09.570545+0000","last_became_active":"2024-04-25T14:59:45.315070+0000","last_became_peered":"2024-04-25T14:59:45.315070+0000","last_unstale":"2024-04-25T15:02:09.570545+0000","last_undegraded":"2024-04-25T15:02:09.570545+0000","last_fullsized":"2024-04-25T15:02:09.570545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:32.492027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562228+0000","last_change":"2024-04-25T14:59:46.318218+0000","last_active":"2024-04-25T15:02:09.562228+0000","last_peered":"2024-04-25T15:02:09.562228+0000","last_clean":"2024-04-25T15:02:09.562228+0000","last_became_active":"2024-04-25T14:59:46.317855+0000","last_became_peered":"2024-04-25T14:59:46.317855+0000","last_unstale":"2024-04-25T15:02:09.562228+0000","last_undegraded":"2024-04-25T15:02:09.562228+0000","last_fullsized":"2024-04-25T15:02:09.562228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:11.712691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574051+0000","last_change":"2024-04-25T14:59:45.319624+0000","last_active":"2024-04-25T15:02:09.574051+0000","last_peered":"2024-04-25T15:02:09.574051+0000","last_clean":"2024-04-25T15:02:09.574051+0000","last_became_active":"2024-04-25T14:59:45.319310+0000","last_became_peered":"2024-04-25T14:59:45.319310+0000","last_unstale":"2024-04-25T15:02:09.574051+0000","last_undegraded":"2024-04-25T15:02:09.574051+0000","last_fullsized":"2024-04-25T15:02:09.574051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:40.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570635+0000","last_change":"2024-04-25T14:59:46.317382+0000","last_active":"2024-04-25T15:02:09.570635+0000","last_peered":"2024-04-25T15:02:09.570635+0000","last_clean":"2024-04-25T15:02:09.570635+0000","last_became_active":"2024-04-25T14:59:46.316993+0000","last_became_peered":"2024-04-25T14:59:46.316993+0000","last_unstale":"2024-04-25T15:02:09.570635+0000","last_undegraded":"2024-04-25T15:02:09.570635+0000","last_fullsized":"2024-04-25T15:02:09.570635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:06.618964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143699+0000","last_change":"2024-04-25T14:59:45.322731+0000","last_active":"2024-04-25T15:02:10.143699+0000","last_peered":"2024-04-25T15:02:10.143699+0000","last_clean":"2024-04-25T15:02:10.143699+0000","last_became_active":"2024-04-25T14:59:45.322360+0000","last_became_peered":"2024-04-25T14:59:45.322360+0000","last_unstale":"2024-04-25T15:02:10.143699+0000","last_undegraded":"2024-04-25T15:02:10.143699+0000","last_fullsized":"2024-04-25T15:02:10.143699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:02.890016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562377+0000","last_change":"2024-04-25T14:59:46.327195+0000","last_active":"2024-04-25T15:02:09.562377+0000","last_peered":"2024-04-25T15:02:09.562377+0000","last_clean":"2024-04-25T15:02:09.562377+0000","last_became_active":"2024-04-25T14:59:46.322053+0000","last_became_peered":"2024-04-25T14:59:46.322053+0000","last_unstale":"2024-04-25T15:02:09.562377+0000","last_undegraded":"2024-04-25T15:02:09.562377+0000","last_fullsized":"2024-04-25T15:02:09.562377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:35.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570537+0000","last_change":"2024-04-25T14:59:45.315688+0000","last_active":"2024-04-25T15:02:09.570537+0000","last_peered":"2024-04-25T15:02:09.570537+0000","last_clean":"2024-04-25T15:02:09.570537+0000","last_became_active":"2024-04-25T14:59:45.315074+0000","last_became_peered":"2024-04-25T14:59:45.315074+0000","last_unstale":"2024-04-25T15:02:09.570537+0000","last_undegraded":"2024-04-25T15:02:09.570537+0000","last_fullsized":"2024-04-25T15:02:09.570537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:49.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559470+0000","last_change":"2024-04-25T14:59:46.322874+0000","last_active":"2024-04-25T15:02:09.559470+0000","last_peered":"2024-04-25T15:02:09.559470+0000","last_clean":"2024-04-25T15:02:09.559470+0000","last_became_active":"2024-04-25T14:59:46.322492+0000","last_became_peered":"2024-04-25T14:59:46.322492+0000","last_unstale":"2024-04-25T15:02:09.559470+0000","last_undegraded":"2024-04-25T15:02:09.559470+0000","last_fullsized":"2024-04-25T15:02:09.559470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:21.294792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574373+0000","last_change":"2024-04-25T14:59:45.322448+0000","last_active":"2024-04-25T15:02:09.574373+0000","last_peered":"2024-04-25T15:02:09.574373+0000","last_clean":"2024-04-25T15:02:09.574373+0000","last_became_active":"2024-04-25T14:59:45.322127+0000","last_became_peered":"2024-04-25T14:59:45.322127+0000","last_unstale":"2024-04-25T15:02:09.574373+0000","last_undegraded":"2024-04-25T15:02:09.574373+0000","last_fullsized":"2024-04-25T15:02:09.574373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.284668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740025+0000","last_change":"2024-04-25T14:59:46.318371+0000","last_active":"2024-04-25T15:02:09.740025+0000","last_peered":"2024-04-25T15:02:09.740025+0000","last_clean":"2024-04-25T15:02:09.740025+0000","last_became_active":"2024-04-25T14:59:46.317819+0000","last_became_peered":"2024-04-25T14:59:46.317819+0000","last_unstale":"2024-04-25T15:02:09.740025+0000","last_undegraded":"2024-04-25T15:02:09.740025+0000","last_fullsized":"2024-04-25T15:02:09.740025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.047982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427035+0000","last_change":"2024-04-25T14:59:45.318286+0000","last_active":"2024-04-25T15:02:09.427035+0000","last_peered":"2024-04-25T15:02:09.427035+0000","last_clean":"2024-04-25T15:02:09.427035+0000","last_became_active":"2024-04-25T14:59:45.317787+0000","last_became_peered":"2024-04-25T14:59:45.317787+0000","last_unstale":"2024-04-25T15:02:09.427035+0000","last_undegraded":"2024-04-25T15:02:09.427035+0000","last_fullsized":"2024-04-25T15:02:09.427035+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:06.508164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428089+0000","last_change":"2024-04-25T15:00:02.988627+0000","last_active":"2024-04-25T15:02:09.428089+0000","last_peered":"2024-04-25T15:02:09.428089+0000","last_clean":"2024-04-25T15:02:09.428089+0000","last_became_active":"2024-04-25T15:00:02.987379+0000","last_became_peered":"2024-04-25T15:00:02.987379+0000","last_unstale":"2024-04-25T15:02:09.428089+0000","last_undegraded":"2024-04-25T15:02:09.428089+0000","last_fullsized":"2024-04-25T15:02:09.428089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:38.564177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570880+0000","last_change":"2024-04-25T14:59:45.327951+0000","last_active":"2024-04-25T15:02:09.570880+0000","last_peered":"2024-04-25T15:02:09.570880+0000","last_clean":"2024-04-25T15:02:09.570880+0000","last_became_active":"2024-04-25T14:59:45.327660+0000","last_became_peered":"2024-04-25T14:59:45.327660+0000","last_unstale":"2024-04-25T15:02:09.570880+0000","last_undegraded":"2024-04-25T15:02:09.570880+0000","last_fullsized":"2024-04-25T15:02:09.570880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:29.862798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740151+0000","last_change":"2024-04-25T14:59:46.304038+0000","last_active":"2024-04-25T15:02:09.740151+0000","last_peered":"2024-04-25T15:02:09.740151+0000","last_clean":"2024-04-25T15:02:09.740151+0000","last_became_active":"2024-04-25T14:59:46.303713+0000","last_became_peered":"2024-04-25T14:59:46.303713+0000","last_unstale":"2024-04-25T15:02:09.740151+0000","last_undegraded":"2024-04-25T15:02:09.740151+0000","last_fullsized":"2024-04-25T15:02:09.740151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:30.476591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":347,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.947930+0000","last_change":"2024-04-25T14:59:45.315738+0000","last_active":"2024-04-25T15:02:09.947930+0000","last_peered":"2024-04-25T15:02:09.947930+0000","last_clean":"2024-04-25T15:02:09.947930+0000","last_became_active":"2024-04-25T14:59:45.315423+0000","last_became_peered":"2024-04-25T14:59:45.315423+0000","last_unstale":"2024-04-25T15:02:09.947930+0000","last_undegraded":"2024-04-25T15:02:09.947930+0000","last_fullsized":"2024-04-25T15:02:09.947930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:33.135348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":292,"num_read_kb":212,"num_write":33,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562433+0000","last_change":"2024-04-25T14:59:46.308085+0000","last_active":"2024-04-25T15:02:09.562433+0000","last_peered":"2024-04-25T15:02:09.562433+0000","last_clean":"2024-04-25T15:02:09.562433+0000","last_became_active":"2024-04-25T14:59:46.307739+0000","last_became_peered":"2024-04-25T14:59:46.307739+0000","last_unstale":"2024-04-25T15:02:09.562433+0000","last_undegraded":"2024-04-25T15:02:09.562433+0000","last_fullsized":"2024-04-25T15:02:09.562433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:57.949147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426603+0000","last_change":"2024-04-25T14:59:45.327058+0000","last_active":"2024-04-25T15:02:09.426603+0000","last_peered":"2024-04-25T15:02:09.426603+0000","last_clean":"2024-04-25T15:02:09.426603+0000","last_became_active":"2024-04-25T14:59:45.326817+0000","last_became_peered":"2024-04-25T14:59:45.326817+0000","last_unstale":"2024-04-25T15:02:09.426603+0000","last_undegraded":"2024-04-25T15:02:09.426603+0000","last_fullsized":"2024-04-25T15:02:09.426603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:51.087966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143375+0000","last_change":"2024-04-25T14:59:46.328901+0000","last_active":"2024-04-25T15:02:10.143375+0000","last_peered":"2024-04-25T15:02:10.143375+0000","last_clean":"2024-04-25T15:02:10.143375+0000","last_became_active":"2024-04-25T14:59:46.328496+0000","last_became_peered":"2024-04-25T14:59:46.328496+0000","last_unstale":"2024-04-25T15:02:10.143375+0000","last_undegraded":"2024-04-25T15:02:10.143375+0000","last_fullsized":"2024-04-25T15:02:10.143375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:48.149719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570566+0000","last_change":"2024-04-25T14:59:45.316673+0000","last_active":"2024-04-25T15:02:09.570566+0000","last_peered":"2024-04-25T15:02:09.570566+0000","last_clean":"2024-04-25T15:02:09.570566+0000","last_became_active":"2024-04-25T14:59:45.316280+0000","last_became_peered":"2024-04-25T14:59:45.316280+0000","last_unstale":"2024-04-25T15:02:09.570566+0000","last_undegraded":"2024-04-25T15:02:09.570566+0000","last_fullsized":"2024-04-25T15:02:09.570566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:51.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427119+0000","last_change":"2024-04-25T14:59:46.323821+0000","last_active":"2024-04-25T15:02:09.427119+0000","last_peered":"2024-04-25T15:02:09.427119+0000","last_clean":"2024-04-25T15:02:09.427119+0000","last_became_active":"2024-04-25T14:59:46.323434+0000","last_became_peered":"2024-04-25T14:59:46.323434+0000","last_unstale":"2024-04-25T15:02:09.427119+0000","last_undegraded":"2024-04-25T15:02:09.427119+0000","last_fullsized":"2024-04-25T15:02:09.427119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:20.983270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457421+0000","last_change":"2024-04-25T14:59:45.317953+0000","last_active":"2024-04-25T15:02:09.457421+0000","last_peered":"2024-04-25T15:02:09.457421+0000","last_clean":"2024-04-25T15:02:09.457421+0000","last_became_active":"2024-04-25T14:59:45.317617+0000","last_became_peered":"2024-04-25T14:59:45.317617+0000","last_unstale":"2024-04-25T15:02:09.457421+0000","last_undegraded":"2024-04-25T15:02:09.457421+0000","last_fullsized":"2024-04-25T15:02:09.457421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:44.235492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426566+0000","last_change":"2024-04-25T14:59:46.305929+0000","last_active":"2024-04-25T15:02:09.426566+0000","last_peered":"2024-04-25T15:02:09.426566+0000","last_clean":"2024-04-25T15:02:09.426566+0000","last_became_active":"2024-04-25T14:59:46.305604+0000","last_became_peered":"2024-04-25T14:59:46.305604+0000","last_unstale":"2024-04-25T15:02:09.426566+0000","last_undegraded":"2024-04-25T15:02:09.426566+0000","last_fullsized":"2024-04-25T15:02:09.426566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:37.340009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818759+0000","last_change":"2024-04-25T15:00:02.731349+0000","last_active":"2024-04-25T15:02:09.818759+0000","last_peered":"2024-04-25T15:02:09.818759+0000","last_clean":"2024-04-25T15:02:09.818759+0000","last_became_active":"2024-04-25T15:00:02.723113+0000","last_became_peered":"2024-04-25T15:00:02.723113+0000","last_unstale":"2024-04-25T15:02:09.818759+0000","last_undegraded":"2024-04-25T15:02:09.818759+0000","last_fullsized":"2024-04-25T15:02:09.818759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:35.964461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559688+0000","last_change":"2024-04-25T15:00:03.077923+0000","last_active":"2024-04-25T15:02:09.559688+0000","last_peered":"2024-04-25T15:02:09.559688+0000","last_clean":"2024-04-25T15:02:09.559688+0000","last_became_active":"2024-04-25T15:00:03.076709+0000","last_became_peered":"2024-04-25T15:00:03.076709+0000","last_unstale":"2024-04-25T15:02:09.559688+0000","last_undegraded":"2024-04-25T15:02:09.559688+0000","last_fullsized":"2024-04-25T15:02:09.559688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:22.427477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562656+0000","last_change":"2024-04-25T14:59:45.318291+0000","last_active":"2024-04-25T15:02:09.562656+0000","last_peered":"2024-04-25T15:02:09.562656+0000","last_clean":"2024-04-25T15:02:09.562656+0000","last_became_active":"2024-04-25T14:59:45.317590+0000","last_became_peered":"2024-04-25T14:59:45.317590+0000","last_unstale":"2024-04-25T15:02:09.562656+0000","last_undegraded":"2024-04-25T15:02:09.562656+0000","last_fullsized":"2024-04-25T15:02:09.562656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818949+0000","last_change":"2024-04-25T14:59:46.303503+0000","last_active":"2024-04-25T15:02:09.818949+0000","last_peered":"2024-04-25T15:02:09.818949+0000","last_clean":"2024-04-25T15:02:09.818949+0000","last_became_active":"2024-04-25T14:59:46.303127+0000","last_became_peered":"2024-04-25T14:59:46.303127+0000","last_unstale":"2024-04-25T15:02:09.818949+0000","last_undegraded":"2024-04-25T15:02:09.818949+0000","last_fullsized":"2024-04-25T15:02:09.818949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:46.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559547+0000","last_change":"2024-04-25T14:59:45.330010+0000","last_active":"2024-04-25T15:02:09.559547+0000","last_peered":"2024-04-25T15:02:09.559547+0000","last_clean":"2024-04-25T15:02:09.559547+0000","last_became_active":"2024-04-25T14:59:45.329225+0000","last_became_peered":"2024-04-25T14:59:45.329225+0000","last_unstale":"2024-04-25T15:02:09.559547+0000","last_undegraded":"2024-04-25T15:02:09.559547+0000","last_fullsized":"2024-04-25T15:02:09.559547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:01.623458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574265+0000","last_change":"2024-04-25T14:59:46.320003+0000","last_active":"2024-04-25T15:02:09.574265+0000","last_peered":"2024-04-25T15:02:09.574265+0000","last_clean":"2024-04-25T15:02:09.574265+0000","last_became_active":"2024-04-25T14:59:46.319600+0000","last_became_peered":"2024-04-25T14:59:46.319600+0000","last_unstale":"2024-04-25T15:02:09.574265+0000","last_undegraded":"2024-04-25T15:02:09.574265+0000","last_fullsized":"2024-04-25T15:02:09.574265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.916018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426669+0000","last_change":"2024-04-25T14:59:45.325233+0000","last_active":"2024-04-25T15:02:09.426669+0000","last_peered":"2024-04-25T15:02:09.426669+0000","last_clean":"2024-04-25T15:02:09.426669+0000","last_became_active":"2024-04-25T14:59:45.324826+0000","last_became_peered":"2024-04-25T14:59:45.324826+0000","last_unstale":"2024-04-25T15:02:09.426669+0000","last_undegraded":"2024-04-25T15:02:09.426669+0000","last_fullsized":"2024-04-25T15:02:09.426669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.422559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574196+0000","last_change":"2024-04-25T14:59:46.314552+0000","last_active":"2024-04-25T15:02:09.574196+0000","last_peered":"2024-04-25T15:02:09.574196+0000","last_clean":"2024-04-25T15:02:09.574196+0000","last_became_active":"2024-04-25T14:59:46.314159+0000","last_became_peered":"2024-04-25T14:59:46.314159+0000","last_unstale":"2024-04-25T15:02:09.574196+0000","last_undegraded":"2024-04-25T15:02:09.574196+0000","last_fullsized":"2024-04-25T15:02:09.574196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:06.731960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570613+0000","last_change":"2024-04-25T14:59:46.303354+0000","last_active":"2024-04-25T15:02:09.570613+0000","last_peered":"2024-04-25T15:02:09.570613+0000","last_clean":"2024-04-25T15:02:09.570613+0000","last_became_active":"2024-04-25T14:59:46.302869+0000","last_became_peered":"2024-04-25T14:59:46.302869+0000","last_unstale":"2024-04-25T15:02:09.570613+0000","last_undegraded":"2024-04-25T15:02:09.570613+0000","last_fullsized":"2024-04-25T15:02:09.570613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.796902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143216+0000","last_change":"2024-04-25T14:59:45.336686+0000","last_active":"2024-04-25T15:02:10.143216+0000","last_peered":"2024-04-25T15:02:10.143216+0000","last_clean":"2024-04-25T15:02:10.143216+0000","last_became_active":"2024-04-25T14:59:45.335828+0000","last_became_peered":"2024-04-25T14:59:45.335828+0000","last_unstale":"2024-04-25T15:02:10.143216+0000","last_undegraded":"2024-04-25T15:02:10.143216+0000","last_fullsized":"2024-04-25T15:02:10.143216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.676117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563154+0000","last_change":"2024-04-25T14:59:46.319541+0000","last_active":"2024-04-25T15:02:09.563154+0000","last_peered":"2024-04-25T15:02:09.563154+0000","last_clean":"2024-04-25T15:02:09.563154+0000","last_became_active":"2024-04-25T14:59:46.317922+0000","last_became_peered":"2024-04-25T14:59:46.317922+0000","last_unstale":"2024-04-25T15:02:09.563154+0000","last_undegraded":"2024-04-25T15:02:09.563154+0000","last_fullsized":"2024-04-25T15:02:09.563154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.454075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457383+0000","last_change":"2024-04-25T14:59:45.313817+0000","last_active":"2024-04-25T15:02:09.457383+0000","last_peered":"2024-04-25T15:02:09.457383+0000","last_clean":"2024-04-25T15:02:09.457383+0000","last_became_active":"2024-04-25T14:59:45.313507+0000","last_became_peered":"2024-04-25T14:59:45.313507+0000","last_unstale":"2024-04-25T15:02:09.457383+0000","last_undegraded":"2024-04-25T15:02:09.457383+0000","last_fullsized":"2024-04-25T15:02:09.457383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:18.126351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426622+0000","last_change":"2024-04-25T14:58:29.935018+0000","last_active":"2024-04-25T15:02:09.426622+0000","last_peered":"2024-04-25T15:02:09.426622+0000","last_clean":"2024-04-25T15:02:09.426622+0000","last_became_active":"2024-04-25T14:58:29.934454+0000","last_became_peered":"2024-04-25T14:58:29.934454+0000","last_unstale":"2024-04-25T15:02:09.426622+0000","last_undegraded":"2024-04-25T15:02:09.426622+0000","last_fullsized":"2024-04-25T15:02:09.426622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_clean_scrub_stamp":"2024-04-25T14:54:46.455802+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:09.308783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143505+0000","last_change":"2024-04-25T14:59:46.319680+0000","last_active":"2024-04-25T15:02:10.143505+0000","last_peered":"2024-04-25T15:02:10.143505+0000","last_clean":"2024-04-25T15:02:10.143505+0000","last_became_active":"2024-04-25T14:59:46.319225+0000","last_became_peered":"2024-04-25T14:59:46.319225+0000","last_unstale":"2024-04-25T15:02:10.143505+0000","last_undegraded":"2024-04-25T15:02:10.143505+0000","last_fullsized":"2024-04-25T15:02:10.143505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:34.629244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426789+0000","last_change":"2024-04-25T14:59:45.318087+0000","last_active":"2024-04-25T15:02:09.426789+0000","last_peered":"2024-04-25T15:02:09.426789+0000","last_clean":"2024-04-25T15:02:09.426789+0000","last_became_active":"2024-04-25T14:59:45.317745+0000","last_became_peered":"2024-04-25T14:59:45.317745+0000","last_unstale":"2024-04-25T15:02:09.426789+0000","last_undegraded":"2024-04-25T15:02:09.426789+0000","last_fullsized":"2024-04-25T15:02:09.426789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:17.202201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562427+0000","last_change":"2024-04-25T14:59:45.332358+0000","last_active":"2024-04-25T15:02:09.562427+0000","last_peered":"2024-04-25T15:02:09.562427+0000","last_clean":"2024-04-25T15:02:09.562427+0000","last_became_active":"2024-04-25T14:59:45.332089+0000","last_became_peered":"2024-04-25T14:59:45.332089+0000","last_unstale":"2024-04-25T15:02:09.562427+0000","last_undegraded":"2024-04-25T15:02:09.562427+0000","last_fullsized":"2024-04-25T15:02:09.562427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:50.937548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143337+0000","last_change":"2024-04-25T14:59:46.320945+0000","last_active":"2024-04-25T15:02:10.143337+0000","last_peered":"2024-04-25T15:02:10.143337+0000","last_clean":"2024-04-25T15:02:10.143337+0000","last_became_active":"2024-04-25T14:59:46.320602+0000","last_became_peered":"2024-04-25T14:59:46.320602+0000","last_unstale":"2024-04-25T15:02:10.143337+0000","last_undegraded":"2024-04-25T15:02:10.143337+0000","last_fullsized":"2024-04-25T15:02:10.143337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:32.282647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'42","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.694772+0000","last_change":"2024-04-25T14:59:45.334325+0000","last_active":"2024-04-25T15:02:09.694772+0000","last_peered":"2024-04-25T15:02:09.694772+0000","last_clean":"2024-04-25T15:02:09.694772+0000","last_became_active":"2024-04-25T14:59:45.333969+0000","last_became_peered":"2024-04-25T14:59:45.333969+0000","last_unstale":"2024-04-25T15:02:09.694772+0000","last_undegraded":"2024-04-25T15:02:09.694772+0000","last_fullsized":"2024-04-25T15:02:09.694772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:02.694267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":75,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":18,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559723+0000","last_change":"2024-04-25T15:00:02.726558+0000","last_active":"2024-04-25T15:02:09.559723+0000","last_peered":"2024-04-25T15:02:09.559723+0000","last_clean":"2024-04-25T15:02:09.559723+0000","last_became_active":"2024-04-25T15:00:02.725467+0000","last_became_peered":"2024-04-25T15:00:02.725467+0000","last_unstale":"2024-04-25T15:02:09.559723+0000","last_undegraded":"2024-04-25T15:02:09.559723+0000","last_fullsized":"2024-04-25T15:02:09.559723+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.508211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457534+0000","last_change":"2024-04-25T14:59:45.325553+0000","last_active":"2024-04-25T15:02:09.457534+0000","last_peered":"2024-04-25T15:02:09.457534+0000","last_clean":"2024-04-25T15:02:09.457534+0000","last_became_active":"2024-04-25T14:59:45.325201+0000","last_became_peered":"2024-04-25T14:59:45.325201+0000","last_unstale":"2024-04-25T15:02:09.457534+0000","last_undegraded":"2024-04-25T15:02:09.457534+0000","last_fullsized":"2024-04-25T15:02:09.457534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:05.068269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427983+0000","last_change":"2024-04-25T14:59:46.318252+0000","last_active":"2024-04-25T15:02:09.427983+0000","last_peered":"2024-04-25T15:02:09.427983+0000","last_clean":"2024-04-25T15:02:09.427983+0000","last_became_active":"2024-04-25T14:59:46.317601+0000","last_became_peered":"2024-04-25T14:59:46.317601+0000","last_unstale":"2024-04-25T15:02:09.427983+0000","last_undegraded":"2024-04-25T15:02:09.427983+0000","last_fullsized":"2024-04-25T15:02:09.427983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:33.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.739599+0000","last_change":"2024-04-25T14:59:45.329430+0000","last_active":"2024-04-25T15:02:09.739599+0000","last_peered":"2024-04-25T15:02:09.739599+0000","last_clean":"2024-04-25T15:02:09.739599+0000","last_became_active":"2024-04-25T14:59:45.329140+0000","last_became_peered":"2024-04-25T14:59:45.329140+0000","last_unstale":"2024-04-25T15:02:09.739599+0000","last_undegraded":"2024-04-25T15:02:09.739599+0000","last_fullsized":"2024-04-25T15:02:09.739599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:49.114820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562082+0000","last_change":"2024-04-25T14:59:46.299821+0000","last_active":"2024-04-25T15:02:09.562082+0000","last_peered":"2024-04-25T15:02:09.562082+0000","last_clean":"2024-04-25T15:02:09.562082+0000","last_became_active":"2024-04-25T14:59:46.299511+0000","last_became_peered":"2024-04-25T14:59:46.299511+0000","last_unstale":"2024-04-25T15:02:09.562082+0000","last_undegraded":"2024-04-25T15:02:09.562082+0000","last_fullsized":"2024-04-25T15:02:09.562082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:23.409333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562929+0000","last_change":"2024-04-25T14:59:45.331464+0000","last_active":"2024-04-25T15:02:09.562929+0000","last_peered":"2024-04-25T15:02:09.562929+0000","last_clean":"2024-04-25T15:02:09.562929+0000","last_became_active":"2024-04-25T14:59:45.327879+0000","last_became_peered":"2024-04-25T14:59:45.327879+0000","last_unstale":"2024-04-25T15:02:09.562929+0000","last_undegraded":"2024-04-25T15:02:09.562929+0000","last_fullsized":"2024-04-25T15:02:09.562929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:50.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818825+0000","last_change":"2024-04-25T14:59:46.310621+0000","last_active":"2024-04-25T15:02:09.818825+0000","last_peered":"2024-04-25T15:02:09.818825+0000","last_clean":"2024-04-25T15:02:09.818825+0000","last_became_active":"2024-04-25T14:59:46.310292+0000","last_became_peered":"2024-04-25T14:59:46.310292+0000","last_unstale":"2024-04-25T15:02:09.818825+0000","last_undegraded":"2024-04-25T15:02:09.818825+0000","last_fullsized":"2024-04-25T15:02:09.818825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:20.035121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":13,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818862+0000","last_change":"2024-04-25T15:00:02.713741+0000","last_active":"2024-04-25T15:02:09.818862+0000","last_peered":"2024-04-25T15:02:09.818862+0000","last_clean":"2024-04-25T15:02:09.818862+0000","last_became_active":"2024-04-25T15:00:02.713336+0000","last_became_peered":"2024-04-25T15:00:02.713336+0000","last_unstale":"2024-04-25T15:02:09.818862+0000","last_undegraded":"2024-04-25T15:02:09.818862+0000","last_fullsized":"2024-04-25T15:02:09.818862+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:37.967019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563077+0000","last_change":"2024-04-25T14:59:46.326568+0000","last_active":"2024-04-25T15:02:09.563077+0000","last_peered":"2024-04-25T15:02:09.563077+0000","last_clean":"2024-04-25T15:02:09.563077+0000","last_became_active":"2024-04-25T14:59:46.324019+0000","last_became_peered":"2024-04-25T14:59:46.324019+0000","last_unstale":"2024-04-25T15:02:09.563077+0000","last_undegraded":"2024-04-25T15:02:09.563077+0000","last_fullsized":"2024-04-25T15:02:09.563077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562234+0000","last_change":"2024-04-25T14:59:45.316935+0000","last_active":"2024-04-25T15:02:09.562234+0000","last_peered":"2024-04-25T15:02:09.562234+0000","last_clean":"2024-04-25T15:02:09.562234+0000","last_became_active":"2024-04-25T14:59:45.316604+0000","last_became_peered":"2024-04-25T14:59:45.316604+0000","last_unstale":"2024-04-25T15:02:09.562234+0000","last_undegraded":"2024-04-25T15:02:09.562234+0000","last_fullsized":"2024-04-25T15:02:09.562234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:45.699926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426847+0000","last_change":"2024-04-25T14:59:46.316195+0000","last_active":"2024-04-25T15:02:09.426847+0000","last_peered":"2024-04-25T15:02:09.426847+0000","last_clean":"2024-04-25T15:02:09.426847+0000","last_became_active":"2024-04-25T14:59:46.315809+0000","last_became_peered":"2024-04-25T14:59:46.315809+0000","last_unstale":"2024-04-25T15:02:09.426847+0000","last_undegraded":"2024-04-25T15:02:09.426847+0000","last_fullsized":"2024-04-25T15:02:09.426847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:34.440058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426952+0000","last_change":"2024-04-25T14:59:45.325137+0000","last_active":"2024-04-25T15:02:09.426952+0000","last_peered":"2024-04-25T15:02:09.426952+0000","last_clean":"2024-04-25T15:02:09.426952+0000","last_became_active":"2024-04-25T14:59:45.324769+0000","last_became_peered":"2024-04-25T14:59:45.324769+0000","last_unstale":"2024-04-25T15:02:09.426952+0000","last_undegraded":"2024-04-25T15:02:09.426952+0000","last_fullsized":"2024-04-25T15:02:09.426952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:02.778853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427258+0000","last_change":"2024-04-25T14:59:46.304099+0000","last_active":"2024-04-25T15:02:09.427258+0000","last_peered":"2024-04-25T15:02:09.427258+0000","last_clean":"2024-04-25T15:02:09.427258+0000","last_became_active":"2024-04-25T14:59:46.303774+0000","last_became_peered":"2024-04-25T14:59:46.303774+0000","last_unstale":"2024-04-25T15:02:09.427258+0000","last_undegraded":"2024-04-25T15:02:09.427258+0000","last_fullsized":"2024-04-25T15:02:09.427258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.308212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.696209+0000","last_change":"2024-04-25T14:59:45.322324+0000","last_active":"2024-04-25T15:02:09.696209+0000","last_peered":"2024-04-25T15:02:09.696209+0000","last_clean":"2024-04-25T15:02:09.696209+0000","last_became_active":"2024-04-25T14:59:45.322031+0000","last_became_peered":"2024-04-25T14:59:45.322031+0000","last_unstale":"2024-04-25T15:02:09.696209+0000","last_undegraded":"2024-04-25T15:02:09.696209+0000","last_fullsized":"2024-04-25T15:02:09.696209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.424928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427253+0000","last_change":"2024-04-25T15:00:02.730051+0000","last_active":"2024-04-25T15:02:09.427253+0000","last_peered":"2024-04-25T15:02:09.427253+0000","last_clean":"2024-04-25T15:02:09.427253+0000","last_became_active":"2024-04-25T15:00:02.725610+0000","last_became_peered":"2024-04-25T15:00:02.725610+0000","last_unstale":"2024-04-25T15:02:09.427253+0000","last_undegraded":"2024-04-25T15:02:09.427253+0000","last_fullsized":"2024-04-25T15:02:09.427253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:04.598366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143595+0000","last_change":"2024-04-25T14:59:45.335502+0000","last_active":"2024-04-25T15:02:10.143595+0000","last_peered":"2024-04-25T15:02:10.143595+0000","last_clean":"2024-04-25T15:02:10.143595+0000","last_became_active":"2024-04-25T14:59:45.335018+0000","last_became_peered":"2024-04-25T14:59:45.335018+0000","last_unstale":"2024-04-25T15:02:10.143595+0000","last_undegraded":"2024-04-25T15:02:10.143595+0000","last_fullsized":"2024-04-25T15:02:10.143595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:11.918001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427140+0000","last_change":"2024-04-25T14:59:46.304766+0000","last_active":"2024-04-25T15:02:09.427140+0000","last_peered":"2024-04-25T15:02:09.427140+0000","last_clean":"2024-04-25T15:02:09.427140+0000","last_became_active":"2024-04-25T14:59:46.304413+0000","last_became_peered":"2024-04-25T14:59:46.304413+0000","last_unstale":"2024-04-25T15:02:09.427140+0000","last_undegraded":"2024-04-25T15:02:09.427140+0000","last_fullsized":"2024-04-25T15:02:09.427140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818984+0000","last_change":"2024-04-25T14:59:46.305285+0000","last_active":"2024-04-25T15:02:09.818984+0000","last_peered":"2024-04-25T15:02:09.818984+0000","last_clean":"2024-04-25T15:02:09.818984+0000","last_became_active":"2024-04-25T14:59:46.304934+0000","last_became_peered":"2024-04-25T14:59:46.304934+0000","last_unstale":"2024-04-25T15:02:09.818984+0000","last_undegraded":"2024-04-25T15:02:09.818984+0000","last_fullsized":"2024-04-25T15:02:09.818984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:44.892921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426447+0000","last_change":"2024-04-25T15:00:02.718240+0000","last_active":"2024-04-25T15:02:09.426447+0000","last_peered":"2024-04-25T15:02:09.426447+0000","last_clean":"2024-04-25T15:02:09.426447+0000","last_became_active":"2024-04-25T15:00:02.717792+0000","last_became_peered":"2024-04-25T15:00:02.717792+0000","last_unstale":"2024-04-25T15:02:09.426447+0000","last_undegraded":"2024-04-25T15:02:09.426447+0000","last_fullsized":"2024-04-25T15:02:09.426447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:49.532532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426395+0000","last_change":"2024-04-25T14:59:46.319694+0000","last_active":"2024-04-25T15:02:09.426395+0000","last_peered":"2024-04-25T15:02:09.426395+0000","last_clean":"2024-04-25T15:02:09.426395+0000","last_became_active":"2024-04-25T14:59:46.319417+0000","last_became_peered":"2024-04-25T14:59:46.319417+0000","last_unstale":"2024-04-25T15:02:09.426395+0000","last_undegraded":"2024-04-25T15:02:09.426395+0000","last_fullsized":"2024-04-25T15:02:09.426395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.177498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428062+0000","last_change":"2024-04-25T15:00:02.986849+0000","last_active":"2024-04-25T15:02:09.428062+0000","last_peered":"2024-04-25T15:02:09.428062+0000","last_clean":"2024-04-25T15:02:09.428062+0000","last_became_active":"2024-04-25T15:00:02.986345+0000","last_became_peered":"2024-04-25T15:00:02.986345+0000","last_unstale":"2024-04-25T15:02:09.428062+0000","last_undegraded":"2024-04-25T15:02:09.428062+0000","last_fullsized":"2024-04-25T15:02:09.428062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:22.707469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143409+0000","last_change":"2024-04-25T14:59:46.319167+0000","last_active":"2024-04-25T15:02:10.143409+0000","last_peered":"2024-04-25T15:02:10.143409+0000","last_clean":"2024-04-25T15:02:10.143409+0000","last_became_active":"2024-04-25T14:59:46.318804+0000","last_became_peered":"2024-04-25T14:59:46.318804+0000","last_unstale":"2024-04-25T15:02:10.143409+0000","last_undegraded":"2024-04-25T15:02:10.143409+0000","last_fullsized":"2024-04-25T15:02:10.143409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:34.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818946+0000","last_change":"2024-04-25T14:59:46.304554+0000","last_active":"2024-04-25T15:02:09.818946+0000","last_peered":"2024-04-25T15:02:09.818946+0000","last_clean":"2024-04-25T15:02:09.818946+0000","last_became_active":"2024-04-25T14:59:46.304062+0000","last_became_peered":"2024-04-25T14:59:46.304062+0000","last_unstale":"2024-04-25T15:02:09.818946+0000","last_undegraded":"2024-04-25T15:02:09.818946+0000","last_fullsized":"2024-04-25T15:02:09.818946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426897+0000","last_change":"2024-04-25T14:59:46.299104+0000","last_active":"2024-04-25T15:02:09.426897+0000","last_peered":"2024-04-25T15:02:09.426897+0000","last_clean":"2024-04-25T15:02:09.426897+0000","last_became_active":"2024-04-25T14:59:46.298765+0000","last_became_peered":"2024-04-25T14:59:46.298765+0000","last_unstale":"2024-04-25T15:02:09.426897+0000","last_undegraded":"2024-04-25T15:02:09.426897+0000","last_fullsized":"2024-04-25T15:02:09.426897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.494298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563297+0000","last_change":"2024-04-25T14:59:46.328885+0000","last_active":"2024-04-25T15:02:09.563297+0000","last_peered":"2024-04-25T15:02:09.563297+0000","last_clean":"2024-04-25T15:02:09.563297+0000","last_became_active":"2024-04-25T14:59:46.328375+0000","last_became_peered":"2024-04-25T14:59:46.328375+0000","last_unstale":"2024-04-25T15:02:09.563297+0000","last_undegraded":"2024-04-25T15:02:09.563297+0000","last_fullsized":"2024-04-25T15:02:09.563297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:08.959045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570545+0000","last_change":"2024-04-25T14:59:46.317403+0000","last_active":"2024-04-25T15:02:09.570545+0000","last_peered":"2024-04-25T15:02:09.570545+0000","last_clean":"2024-04-25T15:02:09.570545+0000","last_became_active":"2024-04-25T14:59:46.317100+0000","last_became_peered":"2024-04-25T14:59:46.317100+0000","last_unstale":"2024-04-25T15:02:09.570545+0000","last_undegraded":"2024-04-25T15:02:09.570545+0000","last_fullsized":"2024-04-25T15:02:09.570545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:23.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563030+0000","last_change":"2024-04-25T14:59:46.327147+0000","last_active":"2024-04-25T15:02:09.563030+0000","last_peered":"2024-04-25T15:02:09.563030+0000","last_clean":"2024-04-25T15:02:09.563030+0000","last_became_active":"2024-04-25T14:59:46.321857+0000","last_became_peered":"2024-04-25T14:59:46.321857+0000","last_unstale":"2024-04-25T15:02:09.563030+0000","last_undegraded":"2024-04-25T15:02:09.563030+0000","last_fullsized":"2024-04-25T15:02:09.563030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:46.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740053+0000","last_change":"2024-04-25T14:59:46.310841+0000","last_active":"2024-04-25T15:02:09.740053+0000","last_peered":"2024-04-25T15:02:09.740053+0000","last_clean":"2024-04-25T15:02:09.740053+0000","last_became_active":"2024-04-25T14:59:46.310446+0000","last_became_peered":"2024-04-25T14:59:46.310446+0000","last_unstale":"2024-04-25T15:02:09.740053+0000","last_undegraded":"2024-04-25T15:02:09.740053+0000","last_fullsized":"2024-04-25T15:02:09.740053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.594784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143711+0000","last_change":"2024-04-25T14:59:46.320715+0000","last_active":"2024-04-25T15:02:10.143711+0000","last_peered":"2024-04-25T15:02:10.143711+0000","last_clean":"2024-04-25T15:02:10.143711+0000","last_became_active":"2024-04-25T14:59:46.320238+0000","last_became_peered":"2024-04-25T14:59:46.320238+0000","last_unstale":"2024-04-25T15:02:10.143711+0000","last_undegraded":"2024-04-25T15:02:10.143711+0000","last_fullsized":"2024-04-25T15:02:10.143711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:55.387814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818885+0000","last_change":"2024-04-25T14:59:46.303772+0000","last_active":"2024-04-25T15:02:09.818885+0000","last_peered":"2024-04-25T15:02:09.818885+0000","last_clean":"2024-04-25T15:02:09.818885+0000","last_became_active":"2024-04-25T14:59:46.303435+0000","last_became_peered":"2024-04-25T14:59:46.303435+0000","last_unstale":"2024-04-25T15:02:09.818885+0000","last_undegraded":"2024-04-25T15:02:09.818885+0000","last_fullsized":"2024-04-25T15:02:09.818885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:06.692979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":593374,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":390,"num_read_kb":297,"num_write":194,"num_write_kb":1402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1780122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":467709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28768,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716384,"statfs":{"total":95995035648,"available":95965577216,"internally_reserved":0,"allocated":1794048,"data_stored":1257748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776179,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36680,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708472,"statfs":{"total":95995035648,"available":95957475328,"internally_reserved":0,"allocated":1372160,"data_stored":861624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1175552,"data_stored":664921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1572864,"data_stored":1058776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":958464,"data_stored":458870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":520358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":467799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411420,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":535503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1511424,"data_stored":992746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888144,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":467731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1049420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":199554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:02:15.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:02:16.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:15 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:16.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:15 smithi138 ceph-mon[32875]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:16.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:15 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:16.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:15 smithi094 ceph-mon[27862]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:16.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:15 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:16.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:15 smithi137 ceph-mon[32439]: from='client.25222 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:16.406 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:17.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:16 smithi138 ceph-mon[32875]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:17.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:16 smithi094 ceph-mon[27862]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:17.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:16 smithi137 ceph-mon[32439]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T15:02:18.099 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:18.100 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:18.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T15:02:18.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T15:02:18.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T15:02:18.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:18.101 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:17 smithi094 ceph-mon[27862]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:18.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:18.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:18.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T15:02:18.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:18.209 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:17 smithi138 ceph-mon[32875]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:18.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:18.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:18.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: Updating smithi094:/etc/ceph/ceph.conf 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: Updating smithi137:/etc/ceph/ceph.conf 2024-04-25T15:02:18.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T15:02:18.234 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:18.234 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:17 smithi137 ceph-mon[32439]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:18.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.562+0000 7fb057200700 1 -- 172.21.15.94:0/3034647349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 msgr2=0x7fb050102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:18.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.562+0000 7fb057200700 1 --2- 172.21.15.94:0/3034647349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 0x7fb050102f10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009990 tx=0x7fb03c009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.562+0000 7fb057200700 1 -- 172.21.15.94:0/3034647349 shutdown_connections 2024-04-25T15:02:18.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.562+0000 7fb057200700 1 --2- 172.21.15.94:0/3034647349 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb050103450 0x7fb05010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.562+0000 7fb057200700 1 --2- 172.21.15.94:0/3034647349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050108a50 0x7fb050108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.562+0000 7fb057200700 1 --2- 172.21.15.94:0/3034647349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 0x7fb050102f10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.562+0000 7fb057200700 1 -- 172.21.15.94:0/3034647349 >> 172.21.15.94:0/3034647349 conn(0x7fb050079ba0 msgr2=0x7fb050079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:18.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.563+0000 7fb057200700 1 -- 172.21.15.94:0/3034647349 shutdown_connections 2024-04-25T15:02:18.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.563+0000 7fb057200700 1 -- 172.21.15.94:0/3034647349 wait complete. 2024-04-25T15:02:18.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 Processor -- start 2024-04-25T15:02:18.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 -- start start 2024-04-25T15:02:18.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 0x7fb050074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:18.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050103450 0x7fb050075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:18.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb050108a50 0x7fb0500757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:18.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fb050112c00 con 0x7fb050102a90 2024-04-25T15:02:18.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fb050112a80 con 0x7fb050108a50 2024-04-25T15:02:18.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.564+0000 7fb057200700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb050112d80 con 0x7fb050103450 2024-04-25T15:02:18.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050103450 0x7fb050075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:18.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04fe00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 0x7fb050074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:18.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050103450 0x7fb050075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:49942/0 (socket says 172.21.15.94:49942) 2024-04-25T15:02:18.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 -- 172.21.15.94:0/1407427257 learned_addr learned my addr 172.21.15.94:0/1407427257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:18.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb055200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb050108a50 0x7fb0500757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 -- 172.21.15.94:0/1407427257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb050108a50 msgr2=0x7fb0500757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb050108a50 0x7fb0500757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 -- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 msgr2=0x7fb050074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 0x7fb050074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb04f400700 1 -- 172.21.15.94:0/1407427257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0501a76a0 con 0x7fb050103450 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.565+0000 7fb055200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb050108a50 0x7fb0500757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.566+0000 7fb04f400700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050103450 0x7fb050075280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0057f0 tx=0x7fb03c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:18.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.566+0000 7fb04cc00700 1 -- 172.21.15.94:0/1407427257 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c026070 con 0x7fb050103450 2024-04-25T15:02:18.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.566+0000 7fb04cc00700 1 -- 172.21.15.94:0/1407427257 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03c021410 con 0x7fb050103450 2024-04-25T15:02:18.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.566+0000 7fb04cc00700 1 -- 172.21.15.94:0/1407427257 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c017650 con 0x7fb050103450 2024-04-25T15:02:18.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.566+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0501a7930 con 0x7fb050103450 2024-04-25T15:02:18.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.566+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0501a7dc0 con 0x7fb050103450 2024-04-25T15:02:18.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.568+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb050069730 con 0x7fb050103450 2024-04-25T15:02:18.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.569+0000 7fb04cc00700 1 -- 172.21.15.94:0/1407427257 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb03c029070 con 0x7fb050103450 2024-04-25T15:02:18.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.571+0000 7fb04cc00700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fb0340a3ec0 0x7fb0340a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:18.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.571+0000 7fb04cc00700 1 -- 172.21.15.94:0/1407427257 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fb03c0d16e0 con 0x7fb050103450 2024-04-25T15:02:18.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.571+0000 7fb04fe00700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fb0340a3ec0 0x7fb0340a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:18.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.572+0000 7fb04fe00700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fb0340a3ec0 0x7fb0340a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb040009be0 tx=0x7fb040009260 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:18.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.574+0000 7fb04cc00700 1 -- 172.21.15.94:0/1407427257 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb03c05dff0 con 0x7fb050103450 2024-04-25T15:02:18.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.802+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb050062e80 con 0x7fb0340a3ec0 2024-04-25T15:02:18.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.813+0000 7fb04cc00700 1 -- 172.21.15.94:0/1407427257 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257687 (secure 0 0 0) 0x7fb050062e80 con 0x7fb0340a3ec0 2024-04-25T15:02:18.817 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:18.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fb0340a3ec0 msgr2=0x7fb0340a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fb0340a3ec0 0x7fb0340a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb040009be0 tx=0x7fb040009260 comp rx=0 tx=0).stop 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050103450 msgr2=0x7fb050075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050103450 0x7fb050075280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0057f0 tx=0x7fb03c00bfd0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 shutdown_connections 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fb0340a3ec0 0x7fb0340a6380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fb050108a50 0x7fb0500757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb050102a90 0x7fb050074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 --2- 172.21.15.94:0/1407427257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb050103450 0x7fb050075280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 >> 172.21.15.94:0/1407427257 conn(0x7fb050079ba0 msgr2=0x7fb0501018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 shutdown_connections 2024-04-25T15:02:18.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:18.820+0000 7fb057200700 1 -- 172.21.15.94:0/1407427257 wait complete. 2024-04-25T15:02:18.823 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T15:02:19.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-04-25T15:02:19.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:19.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:19.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:19.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:19 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.486 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:19 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.618 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-04-25T15:02:17.450776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1380608,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":890,"num_read_kb":862,"num_write":525,"num_write_kb":5257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381732,"kb_used_data":15552,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560092,"statfs":{"total":1151940427776,"available":1151549534208,"internally_reserved":0,"allocated":15925248,"data_stored":9754332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":364,"num_read_kb":446,"num_write":148,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.981415"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.819097+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:02:09.819097+0000","last_peered":"2024-04-25T15:02:09.819097+0000","last_clean":"2024-04-25T15:02:09.819097+0000","last_became_active":"2024-04-25T14:59:46.311856+0000","last_became_peered":"2024-04-25T14:59:46.311856+0000","last_unstale":"2024-04-25T15:02:09.819097+0000","last_undegraded":"2024-04-25T15:02:09.819097+0000","last_fullsized":"2024-04-25T15:02:09.819097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:07.469645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427851+0000","last_change":"2024-04-25T14:59:46.310292+0000","last_active":"2024-04-25T15:02:09.427851+0000","last_peered":"2024-04-25T15:02:09.427851+0000","last_clean":"2024-04-25T15:02:09.427851+0000","last_became_active":"2024-04-25T14:59:46.309812+0000","last_became_peered":"2024-04-25T14:59:46.309812+0000","last_unstale":"2024-04-25T15:02:09.427851+0000","last_undegraded":"2024-04-25T15:02:09.427851+0000","last_fullsized":"2024-04-25T15:02:09.427851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:52.448406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428156+0000","last_change":"2024-04-25T14:59:46.300987+0000","last_active":"2024-04-25T15:02:09.428156+0000","last_peered":"2024-04-25T15:02:09.428156+0000","last_clean":"2024-04-25T15:02:09.428156+0000","last_became_active":"2024-04-25T14:59:46.299866+0000","last_became_peered":"2024-04-25T14:59:46.299866+0000","last_unstale":"2024-04-25T15:02:09.428156+0000","last_undegraded":"2024-04-25T15:02:09.428156+0000","last_fullsized":"2024-04-25T15:02:09.428156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:09.831131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457562+0000","last_change":"2024-04-25T14:59:46.304278+0000","last_active":"2024-04-25T15:02:09.457562+0000","last_peered":"2024-04-25T15:02:09.457562+0000","last_clean":"2024-04-25T15:02:09.457562+0000","last_became_active":"2024-04-25T14:59:46.303956+0000","last_became_peered":"2024-04-25T14:59:46.303956+0000","last_unstale":"2024-04-25T15:02:09.457562+0000","last_undegraded":"2024-04-25T15:02:09.457562+0000","last_fullsized":"2024-04-25T15:02:09.457562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.752548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562299+0000","last_change":"2024-04-25T14:59:46.323730+0000","last_active":"2024-04-25T15:02:09.562299+0000","last_peered":"2024-04-25T15:02:09.562299+0000","last_clean":"2024-04-25T15:02:09.562299+0000","last_became_active":"2024-04-25T14:59:46.323383+0000","last_became_peered":"2024-04-25T14:59:46.323383+0000","last_unstale":"2024-04-25T15:02:09.562299+0000","last_undegraded":"2024-04-25T15:02:09.562299+0000","last_fullsized":"2024-04-25T15:02:09.562299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.846392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574191+0000","last_change":"2024-04-25T14:59:46.317623+0000","last_active":"2024-04-25T15:02:09.574191+0000","last_peered":"2024-04-25T15:02:09.574191+0000","last_clean":"2024-04-25T15:02:09.574191+0000","last_became_active":"2024-04-25T14:59:46.317255+0000","last_became_peered":"2024-04-25T14:59:46.317255+0000","last_unstale":"2024-04-25T15:02:09.574191+0000","last_undegraded":"2024-04-25T15:02:09.574191+0000","last_fullsized":"2024-04-25T15:02:09.574191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:24.449615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426641+0000","last_change":"2024-04-25T15:00:02.723289+0000","last_active":"2024-04-25T15:02:09.426641+0000","last_peered":"2024-04-25T15:02:09.426641+0000","last_clean":"2024-04-25T15:02:09.426641+0000","last_became_active":"2024-04-25T15:00:02.722968+0000","last_became_peered":"2024-04-25T15:00:02.722968+0000","last_unstale":"2024-04-25T15:02:09.426641+0000","last_undegraded":"2024-04-25T15:02:09.426641+0000","last_fullsized":"2024-04-25T15:02:09.426641+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:45.674905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562263+0000","last_change":"2024-04-25T14:59:46.327687+0000","last_active":"2024-04-25T15:02:09.562263+0000","last_peered":"2024-04-25T15:02:09.562263+0000","last_clean":"2024-04-25T15:02:09.562263+0000","last_became_active":"2024-04-25T14:59:46.325905+0000","last_became_peered":"2024-04-25T14:59:46.325905+0000","last_unstale":"2024-04-25T15:02:09.562263+0000","last_undegraded":"2024-04-25T15:02:09.562263+0000","last_fullsized":"2024-04-25T15:02:09.562263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:50.058801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740112+0000","last_change":"2024-04-25T14:59:46.304722+0000","last_active":"2024-04-25T15:02:09.740112+0000","last_peered":"2024-04-25T15:02:09.740112+0000","last_clean":"2024-04-25T15:02:09.740112+0000","last_became_active":"2024-04-25T14:59:46.304391+0000","last_became_peered":"2024-04-25T14:59:46.304391+0000","last_unstale":"2024-04-25T15:02:09.740112+0000","last_undegraded":"2024-04-25T15:02:09.740112+0000","last_fullsized":"2024-04-25T15:02:09.740112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:15.735670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574449+0000","last_change":"2024-04-25T14:59:46.307796+0000","last_active":"2024-04-25T15:02:09.574449+0000","last_peered":"2024-04-25T15:02:09.574449+0000","last_clean":"2024-04-25T15:02:09.574449+0000","last_became_active":"2024-04-25T14:59:46.306741+0000","last_became_peered":"2024-04-25T14:59:46.306741+0000","last_unstale":"2024-04-25T15:02:09.574449+0000","last_undegraded":"2024-04-25T15:02:09.574449+0000","last_fullsized":"2024-04-25T15:02:09.574449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:34.407272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427124+0000","last_change":"2024-04-25T14:59:46.306390+0000","last_active":"2024-04-25T15:02:09.427124+0000","last_peered":"2024-04-25T15:02:09.427124+0000","last_clean":"2024-04-25T15:02:09.427124+0000","last_became_active":"2024-04-25T14:59:46.306026+0000","last_became_peered":"2024-04-25T14:59:46.306026+0000","last_unstale":"2024-04-25T15:02:09.427124+0000","last_undegraded":"2024-04-25T15:02:09.427124+0000","last_fullsized":"2024-04-25T15:02:09.427124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:59.340382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457480+0000","last_change":"2024-04-25T14:59:46.305840+0000","last_active":"2024-04-25T15:02:09.457480+0000","last_peered":"2024-04-25T15:02:09.457480+0000","last_clean":"2024-04-25T15:02:09.457480+0000","last_became_active":"2024-04-25T14:59:46.304867+0000","last_became_peered":"2024-04-25T14:59:46.304867+0000","last_unstale":"2024-04-25T15:02:09.457480+0000","last_undegraded":"2024-04-25T15:02:09.457480+0000","last_fullsized":"2024-04-25T15:02:09.457480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:58.569011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143755+0000","last_change":"2024-04-25T14:59:46.318044+0000","last_active":"2024-04-25T15:02:10.143755+0000","last_peered":"2024-04-25T15:02:10.143755+0000","last_clean":"2024-04-25T15:02:10.143755+0000","last_became_active":"2024-04-25T14:59:46.315935+0000","last_became_peered":"2024-04-25T14:59:46.315935+0000","last_unstale":"2024-04-25T15:02:10.143755+0000","last_undegraded":"2024-04-25T15:02:10.143755+0000","last_fullsized":"2024-04-25T15:02:10.143755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:42.366802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427570+0000","last_change":"2024-04-25T14:59:46.308565+0000","last_active":"2024-04-25T15:02:09.427570+0000","last_peered":"2024-04-25T15:02:09.427570+0000","last_clean":"2024-04-25T15:02:09.427570+0000","last_became_active":"2024-04-25T14:59:46.306198+0000","last_became_peered":"2024-04-25T14:59:46.306198+0000","last_unstale":"2024-04-25T15:02:09.427570+0000","last_undegraded":"2024-04-25T15:02:09.427570+0000","last_fullsized":"2024-04-25T15:02:09.427570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:12.686162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427475+0000","last_change":"2024-04-25T14:59:46.308313+0000","last_active":"2024-04-25T15:02:09.427475+0000","last_peered":"2024-04-25T15:02:09.427475+0000","last_clean":"2024-04-25T15:02:09.427475+0000","last_became_active":"2024-04-25T14:59:46.305974+0000","last_became_peered":"2024-04-25T14:59:46.305974+0000","last_unstale":"2024-04-25T15:02:09.427475+0000","last_undegraded":"2024-04-25T15:02:09.427475+0000","last_fullsized":"2024-04-25T15:02:09.427475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.841983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.739661+0000","last_change":"2024-04-25T14:59:46.302726+0000","last_active":"2024-04-25T15:02:09.739661+0000","last_peered":"2024-04-25T15:02:09.739661+0000","last_clean":"2024-04-25T15:02:09.739661+0000","last_became_active":"2024-04-25T14:59:46.302406+0000","last_became_peered":"2024-04-25T14:59:46.302406+0000","last_unstale":"2024-04-25T15:02:09.739661+0000","last_undegraded":"2024-04-25T15:02:09.739661+0000","last_fullsized":"2024-04-25T15:02:09.739661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.819221+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:02:09.819221+0000","last_peered":"2024-04-25T15:02:09.819221+0000","last_clean":"2024-04-25T15:02:09.819221+0000","last_became_active":"2024-04-25T14:59:46.311822+0000","last_became_peered":"2024-04-25T14:59:46.311822+0000","last_unstale":"2024-04-25T15:02:09.819221+0000","last_undegraded":"2024-04-25T15:02:09.819221+0000","last_fullsized":"2024-04-25T15:02:09.819221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.128537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559804+0000","last_change":"2024-04-25T14:59:46.313816+0000","last_active":"2024-04-25T15:02:09.559804+0000","last_peered":"2024-04-25T15:02:09.559804+0000","last_clean":"2024-04-25T15:02:09.559804+0000","last_became_active":"2024-04-25T14:59:46.313462+0000","last_became_peered":"2024-04-25T14:59:46.313462+0000","last_unstale":"2024-04-25T15:02:09.559804+0000","last_undegraded":"2024-04-25T15:02:09.559804+0000","last_fullsized":"2024-04-25T15:02:09.559804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:29.402113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426726+0000","last_change":"2024-04-25T14:59:46.317972+0000","last_active":"2024-04-25T15:02:09.426726+0000","last_peered":"2024-04-25T15:02:09.426726+0000","last_clean":"2024-04-25T15:02:09.426726+0000","last_became_active":"2024-04-25T14:59:46.317466+0000","last_became_peered":"2024-04-25T14:59:46.317466+0000","last_unstale":"2024-04-25T15:02:09.426726+0000","last_undegraded":"2024-04-25T15:02:09.426726+0000","last_fullsized":"2024-04-25T15:02:09.426726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:03.706663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559858+0000","last_change":"2024-04-25T14:59:45.320752+0000","last_active":"2024-04-25T15:02:09.559858+0000","last_peered":"2024-04-25T15:02:09.559858+0000","last_clean":"2024-04-25T15:02:09.559858+0000","last_became_active":"2024-04-25T14:59:45.319463+0000","last_became_peered":"2024-04-25T14:59:45.319463+0000","last_unstale":"2024-04-25T15:02:09.559858+0000","last_undegraded":"2024-04-25T15:02:09.559858+0000","last_fullsized":"2024-04-25T15:02:09.559858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:52.608899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563574+0000","last_change":"2024-04-25T14:59:46.323802+0000","last_active":"2024-04-25T15:02:09.563574+0000","last_peered":"2024-04-25T15:02:09.563574+0000","last_clean":"2024-04-25T15:02:09.563574+0000","last_became_active":"2024-04-25T14:59:46.323498+0000","last_became_peered":"2024-04-25T14:59:46.323498+0000","last_unstale":"2024-04-25T15:02:09.563574+0000","last_undegraded":"2024-04-25T15:02:09.563574+0000","last_fullsized":"2024-04-25T15:02:09.563574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:38.236936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563729+0000","last_change":"2024-04-25T14:59:45.335156+0000","last_active":"2024-04-25T15:02:09.563729+0000","last_peered":"2024-04-25T15:02:09.563729+0000","last_clean":"2024-04-25T15:02:09.563729+0000","last_became_active":"2024-04-25T14:59:45.333589+0000","last_became_peered":"2024-04-25T14:59:45.333589+0000","last_unstale":"2024-04-25T15:02:09.563729+0000","last_undegraded":"2024-04-25T15:02:09.563729+0000","last_fullsized":"2024-04-25T15:02:09.563729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428190+0000","last_change":"2024-04-25T14:59:46.319165+0000","last_active":"2024-04-25T15:02:09.428190+0000","last_peered":"2024-04-25T15:02:09.428190+0000","last_clean":"2024-04-25T15:02:09.428190+0000","last_became_active":"2024-04-25T14:59:46.318801+0000","last_became_peered":"2024-04-25T14:59:46.318801+0000","last_unstale":"2024-04-25T15:02:09.428190+0000","last_undegraded":"2024-04-25T15:02:09.428190+0000","last_fullsized":"2024-04-25T15:02:09.428190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:01.806099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562565+0000","last_change":"2024-04-25T14:59:45.317956+0000","last_active":"2024-04-25T15:02:09.562565+0000","last_peered":"2024-04-25T15:02:09.562565+0000","last_clean":"2024-04-25T15:02:09.562565+0000","last_became_active":"2024-04-25T14:59:45.317655+0000","last_became_peered":"2024-04-25T14:59:45.317655+0000","last_unstale":"2024-04-25T15:02:09.562565+0000","last_undegraded":"2024-04-25T15:02:09.562565+0000","last_fullsized":"2024-04-25T15:02:09.562565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:59.008411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740141+0000","last_change":"2024-04-25T14:59:46.311659+0000","last_active":"2024-04-25T15:02:09.740141+0000","last_peered":"2024-04-25T15:02:09.740141+0000","last_clean":"2024-04-25T15:02:09.740141+0000","last_became_active":"2024-04-25T14:59:46.311287+0000","last_became_peered":"2024-04-25T14:59:46.311287+0000","last_unstale":"2024-04-25T15:02:09.740141+0000","last_undegraded":"2024-04-25T15:02:09.740141+0000","last_fullsized":"2024-04-25T15:02:09.740141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:29.434223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562402+0000","last_change":"2024-04-25T14:59:45.317658+0000","last_active":"2024-04-25T15:02:09.562402+0000","last_peered":"2024-04-25T15:02:09.562402+0000","last_clean":"2024-04-25T15:02:09.562402+0000","last_became_active":"2024-04-25T14:59:45.317311+0000","last_became_peered":"2024-04-25T14:59:45.317311+0000","last_unstale":"2024-04-25T15:02:09.562402+0000","last_undegraded":"2024-04-25T15:02:09.562402+0000","last_fullsized":"2024-04-25T15:02:09.562402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.602412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570817+0000","last_change":"2024-04-25T14:59:46.305456+0000","last_active":"2024-04-25T15:02:09.570817+0000","last_peered":"2024-04-25T15:02:09.570817+0000","last_clean":"2024-04-25T15:02:09.570817+0000","last_became_active":"2024-04-25T14:59:46.305143+0000","last_became_peered":"2024-04-25T14:59:46.305143+0000","last_unstale":"2024-04-25T15:02:09.570817+0000","last_undegraded":"2024-04-25T15:02:09.570817+0000","last_fullsized":"2024-04-25T15:02:09.570817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:03.941200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570635+0000","last_change":"2024-04-25T14:59:45.315407+0000","last_active":"2024-04-25T15:02:09.570635+0000","last_peered":"2024-04-25T15:02:09.570635+0000","last_clean":"2024-04-25T15:02:09.570635+0000","last_became_active":"2024-04-25T14:59:45.315070+0000","last_became_peered":"2024-04-25T14:59:45.315070+0000","last_unstale":"2024-04-25T15:02:09.570635+0000","last_undegraded":"2024-04-25T15:02:09.570635+0000","last_fullsized":"2024-04-25T15:02:09.570635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:32.492027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562302+0000","last_change":"2024-04-25T14:59:46.318218+0000","last_active":"2024-04-25T15:02:09.562302+0000","last_peered":"2024-04-25T15:02:09.562302+0000","last_clean":"2024-04-25T15:02:09.562302+0000","last_became_active":"2024-04-25T14:59:46.317855+0000","last_became_peered":"2024-04-25T14:59:46.317855+0000","last_unstale":"2024-04-25T15:02:09.562302+0000","last_undegraded":"2024-04-25T15:02:09.562302+0000","last_fullsized":"2024-04-25T15:02:09.562302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:11.712691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574151+0000","last_change":"2024-04-25T14:59:45.319624+0000","last_active":"2024-04-25T15:02:09.574151+0000","last_peered":"2024-04-25T15:02:09.574151+0000","last_clean":"2024-04-25T15:02:09.574151+0000","last_became_active":"2024-04-25T14:59:45.319310+0000","last_became_peered":"2024-04-25T14:59:45.319310+0000","last_unstale":"2024-04-25T15:02:09.574151+0000","last_undegraded":"2024-04-25T15:02:09.574151+0000","last_fullsized":"2024-04-25T15:02:09.574151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:40.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570715+0000","last_change":"2024-04-25T14:59:46.317382+0000","last_active":"2024-04-25T15:02:09.570715+0000","last_peered":"2024-04-25T15:02:09.570715+0000","last_clean":"2024-04-25T15:02:09.570715+0000","last_became_active":"2024-04-25T14:59:46.316993+0000","last_became_peered":"2024-04-25T14:59:46.316993+0000","last_unstale":"2024-04-25T15:02:09.570715+0000","last_undegraded":"2024-04-25T15:02:09.570715+0000","last_fullsized":"2024-04-25T15:02:09.570715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:06.618964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143699+0000","last_change":"2024-04-25T14:59:45.322731+0000","last_active":"2024-04-25T15:02:10.143699+0000","last_peered":"2024-04-25T15:02:10.143699+0000","last_clean":"2024-04-25T15:02:10.143699+0000","last_became_active":"2024-04-25T14:59:45.322360+0000","last_became_peered":"2024-04-25T14:59:45.322360+0000","last_unstale":"2024-04-25T15:02:10.143699+0000","last_undegraded":"2024-04-25T15:02:10.143699+0000","last_fullsized":"2024-04-25T15:02:10.143699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:02.890016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562422+0000","last_change":"2024-04-25T14:59:46.327195+0000","last_active":"2024-04-25T15:02:09.562422+0000","last_peered":"2024-04-25T15:02:09.562422+0000","last_clean":"2024-04-25T15:02:09.562422+0000","last_became_active":"2024-04-25T14:59:46.322053+0000","last_became_peered":"2024-04-25T14:59:46.322053+0000","last_unstale":"2024-04-25T15:02:09.562422+0000","last_undegraded":"2024-04-25T15:02:09.562422+0000","last_fullsized":"2024-04-25T15:02:09.562422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:35.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570537+0000","last_change":"2024-04-25T14:59:45.315688+0000","last_active":"2024-04-25T15:02:09.570537+0000","last_peered":"2024-04-25T15:02:09.570537+0000","last_clean":"2024-04-25T15:02:09.570537+0000","last_became_active":"2024-04-25T14:59:45.315074+0000","last_became_peered":"2024-04-25T14:59:45.315074+0000","last_unstale":"2024-04-25T15:02:09.570537+0000","last_undegraded":"2024-04-25T15:02:09.570537+0000","last_fullsized":"2024-04-25T15:02:09.570537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:49.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559531+0000","last_change":"2024-04-25T14:59:46.322874+0000","last_active":"2024-04-25T15:02:09.559531+0000","last_peered":"2024-04-25T15:02:09.559531+0000","last_clean":"2024-04-25T15:02:09.559531+0000","last_became_active":"2024-04-25T14:59:46.322492+0000","last_became_peered":"2024-04-25T14:59:46.322492+0000","last_unstale":"2024-04-25T15:02:09.559531+0000","last_undegraded":"2024-04-25T15:02:09.559531+0000","last_fullsized":"2024-04-25T15:02:09.559531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:21.294792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574373+0000","last_change":"2024-04-25T14:59:45.322448+0000","last_active":"2024-04-25T15:02:09.574373+0000","last_peered":"2024-04-25T15:02:09.574373+0000","last_clean":"2024-04-25T15:02:09.574373+0000","last_became_active":"2024-04-25T14:59:45.322127+0000","last_became_peered":"2024-04-25T14:59:45.322127+0000","last_unstale":"2024-04-25T15:02:09.574373+0000","last_undegraded":"2024-04-25T15:02:09.574373+0000","last_fullsized":"2024-04-25T15:02:09.574373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.284668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740025+0000","last_change":"2024-04-25T14:59:46.318371+0000","last_active":"2024-04-25T15:02:09.740025+0000","last_peered":"2024-04-25T15:02:09.740025+0000","last_clean":"2024-04-25T15:02:09.740025+0000","last_became_active":"2024-04-25T14:59:46.317819+0000","last_became_peered":"2024-04-25T14:59:46.317819+0000","last_unstale":"2024-04-25T15:02:09.740025+0000","last_undegraded":"2024-04-25T15:02:09.740025+0000","last_fullsized":"2024-04-25T15:02:09.740025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.047982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427082+0000","last_change":"2024-04-25T14:59:45.318286+0000","last_active":"2024-04-25T15:02:09.427082+0000","last_peered":"2024-04-25T15:02:09.427082+0000","last_clean":"2024-04-25T15:02:09.427082+0000","last_became_active":"2024-04-25T14:59:45.317787+0000","last_became_peered":"2024-04-25T14:59:45.317787+0000","last_unstale":"2024-04-25T15:02:09.427082+0000","last_undegraded":"2024-04-25T15:02:09.427082+0000","last_fullsized":"2024-04-25T15:02:09.427082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:06.508164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428089+0000","last_change":"2024-04-25T15:00:02.988627+0000","last_active":"2024-04-25T15:02:09.428089+0000","last_peered":"2024-04-25T15:02:09.428089+0000","last_clean":"2024-04-25T15:02:09.428089+0000","last_became_active":"2024-04-25T15:00:02.987379+0000","last_became_peered":"2024-04-25T15:00:02.987379+0000","last_unstale":"2024-04-25T15:02:09.428089+0000","last_undegraded":"2024-04-25T15:02:09.428089+0000","last_fullsized":"2024-04-25T15:02:09.428089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:38.564177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570880+0000","last_change":"2024-04-25T14:59:45.327951+0000","last_active":"2024-04-25T15:02:09.570880+0000","last_peered":"2024-04-25T15:02:09.570880+0000","last_clean":"2024-04-25T15:02:09.570880+0000","last_became_active":"2024-04-25T14:59:45.327660+0000","last_became_peered":"2024-04-25T14:59:45.327660+0000","last_unstale":"2024-04-25T15:02:09.570880+0000","last_undegraded":"2024-04-25T15:02:09.570880+0000","last_fullsized":"2024-04-25T15:02:09.570880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:29.862798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740151+0000","last_change":"2024-04-25T14:59:46.304038+0000","last_active":"2024-04-25T15:02:09.740151+0000","last_peered":"2024-04-25T15:02:09.740151+0000","last_clean":"2024-04-25T15:02:09.740151+0000","last_became_active":"2024-04-25T14:59:46.303713+0000","last_became_peered":"2024-04-25T14:59:46.303713+0000","last_unstale":"2024-04-25T15:02:09.740151+0000","last_undegraded":"2024-04-25T15:02:09.740151+0000","last_fullsized":"2024-04-25T15:02:09.740151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:30.476591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":357,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:14.956914+0000","last_change":"2024-04-25T14:59:45.315738+0000","last_active":"2024-04-25T15:02:14.956914+0000","last_peered":"2024-04-25T15:02:14.956914+0000","last_clean":"2024-04-25T15:02:14.956914+0000","last_became_active":"2024-04-25T14:59:45.315423+0000","last_became_peered":"2024-04-25T14:59:45.315423+0000","last_unstale":"2024-04-25T15:02:14.956914+0000","last_undegraded":"2024-04-25T15:02:14.956914+0000","last_fullsized":"2024-04-25T15:02:14.956914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:33.135348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":302,"num_read_kb":217,"num_write":33,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562433+0000","last_change":"2024-04-25T14:59:46.308085+0000","last_active":"2024-04-25T15:02:09.562433+0000","last_peered":"2024-04-25T15:02:09.562433+0000","last_clean":"2024-04-25T15:02:09.562433+0000","last_became_active":"2024-04-25T14:59:46.307739+0000","last_became_peered":"2024-04-25T14:59:46.307739+0000","last_unstale":"2024-04-25T15:02:09.562433+0000","last_undegraded":"2024-04-25T15:02:09.562433+0000","last_fullsized":"2024-04-25T15:02:09.562433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:57.949147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426603+0000","last_change":"2024-04-25T14:59:45.327058+0000","last_active":"2024-04-25T15:02:09.426603+0000","last_peered":"2024-04-25T15:02:09.426603+0000","last_clean":"2024-04-25T15:02:09.426603+0000","last_became_active":"2024-04-25T14:59:45.326817+0000","last_became_peered":"2024-04-25T14:59:45.326817+0000","last_unstale":"2024-04-25T15:02:09.426603+0000","last_undegraded":"2024-04-25T15:02:09.426603+0000","last_fullsized":"2024-04-25T15:02:09.426603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:51.087966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143375+0000","last_change":"2024-04-25T14:59:46.328901+0000","last_active":"2024-04-25T15:02:10.143375+0000","last_peered":"2024-04-25T15:02:10.143375+0000","last_clean":"2024-04-25T15:02:10.143375+0000","last_became_active":"2024-04-25T14:59:46.328496+0000","last_became_peered":"2024-04-25T14:59:46.328496+0000","last_unstale":"2024-04-25T15:02:10.143375+0000","last_undegraded":"2024-04-25T15:02:10.143375+0000","last_fullsized":"2024-04-25T15:02:10.143375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:48.149719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570566+0000","last_change":"2024-04-25T14:59:45.316673+0000","last_active":"2024-04-25T15:02:09.570566+0000","last_peered":"2024-04-25T15:02:09.570566+0000","last_clean":"2024-04-25T15:02:09.570566+0000","last_became_active":"2024-04-25T14:59:45.316280+0000","last_became_peered":"2024-04-25T14:59:45.316280+0000","last_unstale":"2024-04-25T15:02:09.570566+0000","last_undegraded":"2024-04-25T15:02:09.570566+0000","last_fullsized":"2024-04-25T15:02:09.570566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:51.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427119+0000","last_change":"2024-04-25T14:59:46.323821+0000","last_active":"2024-04-25T15:02:09.427119+0000","last_peered":"2024-04-25T15:02:09.427119+0000","last_clean":"2024-04-25T15:02:09.427119+0000","last_became_active":"2024-04-25T14:59:46.323434+0000","last_became_peered":"2024-04-25T14:59:46.323434+0000","last_unstale":"2024-04-25T15:02:09.427119+0000","last_undegraded":"2024-04-25T15:02:09.427119+0000","last_fullsized":"2024-04-25T15:02:09.427119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:20.983270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457421+0000","last_change":"2024-04-25T14:59:45.317953+0000","last_active":"2024-04-25T15:02:09.457421+0000","last_peered":"2024-04-25T15:02:09.457421+0000","last_clean":"2024-04-25T15:02:09.457421+0000","last_became_active":"2024-04-25T14:59:45.317617+0000","last_became_peered":"2024-04-25T14:59:45.317617+0000","last_unstale":"2024-04-25T15:02:09.457421+0000","last_undegraded":"2024-04-25T15:02:09.457421+0000","last_fullsized":"2024-04-25T15:02:09.457421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:44.235492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426566+0000","last_change":"2024-04-25T14:59:46.305929+0000","last_active":"2024-04-25T15:02:09.426566+0000","last_peered":"2024-04-25T15:02:09.426566+0000","last_clean":"2024-04-25T15:02:09.426566+0000","last_became_active":"2024-04-25T14:59:46.305604+0000","last_became_peered":"2024-04-25T14:59:46.305604+0000","last_unstale":"2024-04-25T15:02:09.426566+0000","last_undegraded":"2024-04-25T15:02:09.426566+0000","last_fullsized":"2024-04-25T15:02:09.426566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:37.340009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818759+0000","last_change":"2024-04-25T15:00:02.731349+0000","last_active":"2024-04-25T15:02:09.818759+0000","last_peered":"2024-04-25T15:02:09.818759+0000","last_clean":"2024-04-25T15:02:09.818759+0000","last_became_active":"2024-04-25T15:00:02.723113+0000","last_became_peered":"2024-04-25T15:00:02.723113+0000","last_unstale":"2024-04-25T15:02:09.818759+0000","last_undegraded":"2024-04-25T15:02:09.818759+0000","last_fullsized":"2024-04-25T15:02:09.818759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:35.964461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559688+0000","last_change":"2024-04-25T15:00:03.077923+0000","last_active":"2024-04-25T15:02:09.559688+0000","last_peered":"2024-04-25T15:02:09.559688+0000","last_clean":"2024-04-25T15:02:09.559688+0000","last_became_active":"2024-04-25T15:00:03.076709+0000","last_became_peered":"2024-04-25T15:00:03.076709+0000","last_unstale":"2024-04-25T15:02:09.559688+0000","last_undegraded":"2024-04-25T15:02:09.559688+0000","last_fullsized":"2024-04-25T15:02:09.559688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:22.427477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562656+0000","last_change":"2024-04-25T14:59:45.318291+0000","last_active":"2024-04-25T15:02:09.562656+0000","last_peered":"2024-04-25T15:02:09.562656+0000","last_clean":"2024-04-25T15:02:09.562656+0000","last_became_active":"2024-04-25T14:59:45.317590+0000","last_became_peered":"2024-04-25T14:59:45.317590+0000","last_unstale":"2024-04-25T15:02:09.562656+0000","last_undegraded":"2024-04-25T15:02:09.562656+0000","last_fullsized":"2024-04-25T15:02:09.562656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818949+0000","last_change":"2024-04-25T14:59:46.303503+0000","last_active":"2024-04-25T15:02:09.818949+0000","last_peered":"2024-04-25T15:02:09.818949+0000","last_clean":"2024-04-25T15:02:09.818949+0000","last_became_active":"2024-04-25T14:59:46.303127+0000","last_became_peered":"2024-04-25T14:59:46.303127+0000","last_unstale":"2024-04-25T15:02:09.818949+0000","last_undegraded":"2024-04-25T15:02:09.818949+0000","last_fullsized":"2024-04-25T15:02:09.818949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:46.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559547+0000","last_change":"2024-04-25T14:59:45.330010+0000","last_active":"2024-04-25T15:02:09.559547+0000","last_peered":"2024-04-25T15:02:09.559547+0000","last_clean":"2024-04-25T15:02:09.559547+0000","last_became_active":"2024-04-25T14:59:45.329225+0000","last_became_peered":"2024-04-25T14:59:45.329225+0000","last_unstale":"2024-04-25T15:02:09.559547+0000","last_undegraded":"2024-04-25T15:02:09.559547+0000","last_fullsized":"2024-04-25T15:02:09.559547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:01.623458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574265+0000","last_change":"2024-04-25T14:59:46.320003+0000","last_active":"2024-04-25T15:02:09.574265+0000","last_peered":"2024-04-25T15:02:09.574265+0000","last_clean":"2024-04-25T15:02:09.574265+0000","last_became_active":"2024-04-25T14:59:46.319600+0000","last_became_peered":"2024-04-25T14:59:46.319600+0000","last_unstale":"2024-04-25T15:02:09.574265+0000","last_undegraded":"2024-04-25T15:02:09.574265+0000","last_fullsized":"2024-04-25T15:02:09.574265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.916018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426669+0000","last_change":"2024-04-25T14:59:45.325233+0000","last_active":"2024-04-25T15:02:09.426669+0000","last_peered":"2024-04-25T15:02:09.426669+0000","last_clean":"2024-04-25T15:02:09.426669+0000","last_became_active":"2024-04-25T14:59:45.324826+0000","last_became_peered":"2024-04-25T14:59:45.324826+0000","last_unstale":"2024-04-25T15:02:09.426669+0000","last_undegraded":"2024-04-25T15:02:09.426669+0000","last_fullsized":"2024-04-25T15:02:09.426669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.422559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574196+0000","last_change":"2024-04-25T14:59:46.314552+0000","last_active":"2024-04-25T15:02:09.574196+0000","last_peered":"2024-04-25T15:02:09.574196+0000","last_clean":"2024-04-25T15:02:09.574196+0000","last_became_active":"2024-04-25T14:59:46.314159+0000","last_became_peered":"2024-04-25T14:59:46.314159+0000","last_unstale":"2024-04-25T15:02:09.574196+0000","last_undegraded":"2024-04-25T15:02:09.574196+0000","last_fullsized":"2024-04-25T15:02:09.574196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:06.731960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570613+0000","last_change":"2024-04-25T14:59:46.303354+0000","last_active":"2024-04-25T15:02:09.570613+0000","last_peered":"2024-04-25T15:02:09.570613+0000","last_clean":"2024-04-25T15:02:09.570613+0000","last_became_active":"2024-04-25T14:59:46.302869+0000","last_became_peered":"2024-04-25T14:59:46.302869+0000","last_unstale":"2024-04-25T15:02:09.570613+0000","last_undegraded":"2024-04-25T15:02:09.570613+0000","last_fullsized":"2024-04-25T15:02:09.570613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.796902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143216+0000","last_change":"2024-04-25T14:59:45.336686+0000","last_active":"2024-04-25T15:02:10.143216+0000","last_peered":"2024-04-25T15:02:10.143216+0000","last_clean":"2024-04-25T15:02:10.143216+0000","last_became_active":"2024-04-25T14:59:45.335828+0000","last_became_peered":"2024-04-25T14:59:45.335828+0000","last_unstale":"2024-04-25T15:02:10.143216+0000","last_undegraded":"2024-04-25T15:02:10.143216+0000","last_fullsized":"2024-04-25T15:02:10.143216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.676117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563154+0000","last_change":"2024-04-25T14:59:46.319541+0000","last_active":"2024-04-25T15:02:09.563154+0000","last_peered":"2024-04-25T15:02:09.563154+0000","last_clean":"2024-04-25T15:02:09.563154+0000","last_became_active":"2024-04-25T14:59:46.317922+0000","last_became_peered":"2024-04-25T14:59:46.317922+0000","last_unstale":"2024-04-25T15:02:09.563154+0000","last_undegraded":"2024-04-25T15:02:09.563154+0000","last_fullsized":"2024-04-25T15:02:09.563154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.454075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457383+0000","last_change":"2024-04-25T14:59:45.313817+0000","last_active":"2024-04-25T15:02:09.457383+0000","last_peered":"2024-04-25T15:02:09.457383+0000","last_clean":"2024-04-25T15:02:09.457383+0000","last_became_active":"2024-04-25T14:59:45.313507+0000","last_became_peered":"2024-04-25T14:59:45.313507+0000","last_unstale":"2024-04-25T15:02:09.457383+0000","last_undegraded":"2024-04-25T15:02:09.457383+0000","last_fullsized":"2024-04-25T15:02:09.457383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:18.126351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":399,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:11.584390+0000","last_change":"2024-04-25T14:58:29.935018+0000","last_active":"2024-04-25T15:02:11.584390+0000","last_peered":"2024-04-25T15:02:11.584390+0000","last_clean":"2024-04-25T15:02:11.584390+0000","last_became_active":"2024-04-25T14:58:29.934454+0000","last_became_peered":"2024-04-25T14:58:29.934454+0000","last_unstale":"2024-04-25T15:02:11.584390+0000","last_undegraded":"2024-04-25T15:02:11.584390+0000","last_fullsized":"2024-04-25T15:02:11.584390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_clean_scrub_stamp":"2024-04-25T14:54:46.455802+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:09.308783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143505+0000","last_change":"2024-04-25T14:59:46.319680+0000","last_active":"2024-04-25T15:02:10.143505+0000","last_peered":"2024-04-25T15:02:10.143505+0000","last_clean":"2024-04-25T15:02:10.143505+0000","last_became_active":"2024-04-25T14:59:46.319225+0000","last_became_peered":"2024-04-25T14:59:46.319225+0000","last_unstale":"2024-04-25T15:02:10.143505+0000","last_undegraded":"2024-04-25T15:02:10.143505+0000","last_fullsized":"2024-04-25T15:02:10.143505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:34.629244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426789+0000","last_change":"2024-04-25T14:59:45.318087+0000","last_active":"2024-04-25T15:02:09.426789+0000","last_peered":"2024-04-25T15:02:09.426789+0000","last_clean":"2024-04-25T15:02:09.426789+0000","last_became_active":"2024-04-25T14:59:45.317745+0000","last_became_peered":"2024-04-25T14:59:45.317745+0000","last_unstale":"2024-04-25T15:02:09.426789+0000","last_undegraded":"2024-04-25T15:02:09.426789+0000","last_fullsized":"2024-04-25T15:02:09.426789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:17.202201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562427+0000","last_change":"2024-04-25T14:59:45.332358+0000","last_active":"2024-04-25T15:02:09.562427+0000","last_peered":"2024-04-25T15:02:09.562427+0000","last_clean":"2024-04-25T15:02:09.562427+0000","last_became_active":"2024-04-25T14:59:45.332089+0000","last_became_peered":"2024-04-25T14:59:45.332089+0000","last_unstale":"2024-04-25T15:02:09.562427+0000","last_undegraded":"2024-04-25T15:02:09.562427+0000","last_fullsized":"2024-04-25T15:02:09.562427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:50.937548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143337+0000","last_change":"2024-04-25T14:59:46.320945+0000","last_active":"2024-04-25T15:02:10.143337+0000","last_peered":"2024-04-25T15:02:10.143337+0000","last_clean":"2024-04-25T15:02:10.143337+0000","last_became_active":"2024-04-25T14:59:46.320602+0000","last_became_peered":"2024-04-25T14:59:46.320602+0000","last_unstale":"2024-04-25T15:02:10.143337+0000","last_undegraded":"2024-04-25T15:02:10.143337+0000","last_fullsized":"2024-04-25T15:02:10.143337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:32.282647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'96","reported_seq":210,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:12.018657+0000","last_change":"2024-04-25T14:59:45.334325+0000","last_active":"2024-04-25T15:02:12.018657+0000","last_peered":"2024-04-25T15:02:12.018657+0000","last_clean":"2024-04-25T15:02:12.018657+0000","last_became_active":"2024-04-25T14:59:45.333969+0000","last_became_peered":"2024-04-25T14:59:45.333969+0000","last_unstale":"2024-04-25T15:02:12.018657+0000","last_undegraded":"2024-04-25T15:02:12.018657+0000","last_fullsized":"2024-04-25T15:02:12.018657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:02.694267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":176,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559772+0000","last_change":"2024-04-25T15:00:02.726558+0000","last_active":"2024-04-25T15:02:09.559772+0000","last_peered":"2024-04-25T15:02:09.559772+0000","last_clean":"2024-04-25T15:02:09.559772+0000","last_became_active":"2024-04-25T15:00:02.725467+0000","last_became_peered":"2024-04-25T15:00:02.725467+0000","last_unstale":"2024-04-25T15:02:09.559772+0000","last_undegraded":"2024-04-25T15:02:09.559772+0000","last_fullsized":"2024-04-25T15:02:09.559772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.508211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457534+0000","last_change":"2024-04-25T14:59:45.325553+0000","last_active":"2024-04-25T15:02:09.457534+0000","last_peered":"2024-04-25T15:02:09.457534+0000","last_clean":"2024-04-25T15:02:09.457534+0000","last_became_active":"2024-04-25T14:59:45.325201+0000","last_became_peered":"2024-04-25T14:59:45.325201+0000","last_unstale":"2024-04-25T15:02:09.457534+0000","last_undegraded":"2024-04-25T15:02:09.457534+0000","last_fullsized":"2024-04-25T15:02:09.457534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:05.068269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427983+0000","last_change":"2024-04-25T14:59:46.318252+0000","last_active":"2024-04-25T15:02:09.427983+0000","last_peered":"2024-04-25T15:02:09.427983+0000","last_clean":"2024-04-25T15:02:09.427983+0000","last_became_active":"2024-04-25T14:59:46.317601+0000","last_became_peered":"2024-04-25T14:59:46.317601+0000","last_unstale":"2024-04-25T15:02:09.427983+0000","last_undegraded":"2024-04-25T15:02:09.427983+0000","last_fullsized":"2024-04-25T15:02:09.427983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:33.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.739599+0000","last_change":"2024-04-25T14:59:45.329430+0000","last_active":"2024-04-25T15:02:09.739599+0000","last_peered":"2024-04-25T15:02:09.739599+0000","last_clean":"2024-04-25T15:02:09.739599+0000","last_became_active":"2024-04-25T14:59:45.329140+0000","last_became_peered":"2024-04-25T14:59:45.329140+0000","last_unstale":"2024-04-25T15:02:09.739599+0000","last_undegraded":"2024-04-25T15:02:09.739599+0000","last_fullsized":"2024-04-25T15:02:09.739599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:49.114820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562082+0000","last_change":"2024-04-25T14:59:46.299821+0000","last_active":"2024-04-25T15:02:09.562082+0000","last_peered":"2024-04-25T15:02:09.562082+0000","last_clean":"2024-04-25T15:02:09.562082+0000","last_became_active":"2024-04-25T14:59:46.299511+0000","last_became_peered":"2024-04-25T14:59:46.299511+0000","last_unstale":"2024-04-25T15:02:09.562082+0000","last_undegraded":"2024-04-25T15:02:09.562082+0000","last_fullsized":"2024-04-25T15:02:09.562082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:23.409333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562929+0000","last_change":"2024-04-25T14:59:45.331464+0000","last_active":"2024-04-25T15:02:09.562929+0000","last_peered":"2024-04-25T15:02:09.562929+0000","last_clean":"2024-04-25T15:02:09.562929+0000","last_became_active":"2024-04-25T14:59:45.327879+0000","last_became_peered":"2024-04-25T14:59:45.327879+0000","last_unstale":"2024-04-25T15:02:09.562929+0000","last_undegraded":"2024-04-25T15:02:09.562929+0000","last_fullsized":"2024-04-25T15:02:09.562929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:50.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818825+0000","last_change":"2024-04-25T14:59:46.310621+0000","last_active":"2024-04-25T15:02:09.818825+0000","last_peered":"2024-04-25T15:02:09.818825+0000","last_clean":"2024-04-25T15:02:09.818825+0000","last_became_active":"2024-04-25T14:59:46.310292+0000","last_became_peered":"2024-04-25T14:59:46.310292+0000","last_unstale":"2024-04-25T15:02:09.818825+0000","last_undegraded":"2024-04-25T15:02:09.818825+0000","last_fullsized":"2024-04-25T15:02:09.818825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:20.035121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818912+0000","last_change":"2024-04-25T15:00:02.713741+0000","last_active":"2024-04-25T15:02:09.818912+0000","last_peered":"2024-04-25T15:02:09.818912+0000","last_clean":"2024-04-25T15:02:09.818912+0000","last_became_active":"2024-04-25T15:00:02.713336+0000","last_became_peered":"2024-04-25T15:00:02.713336+0000","last_unstale":"2024-04-25T15:02:09.818912+0000","last_undegraded":"2024-04-25T15:02:09.818912+0000","last_fullsized":"2024-04-25T15:02:09.818912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:37.967019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563077+0000","last_change":"2024-04-25T14:59:46.326568+0000","last_active":"2024-04-25T15:02:09.563077+0000","last_peered":"2024-04-25T15:02:09.563077+0000","last_clean":"2024-04-25T15:02:09.563077+0000","last_became_active":"2024-04-25T14:59:46.324019+0000","last_became_peered":"2024-04-25T14:59:46.324019+0000","last_unstale":"2024-04-25T15:02:09.563077+0000","last_undegraded":"2024-04-25T15:02:09.563077+0000","last_fullsized":"2024-04-25T15:02:09.563077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562234+0000","last_change":"2024-04-25T14:59:45.316935+0000","last_active":"2024-04-25T15:02:09.562234+0000","last_peered":"2024-04-25T15:02:09.562234+0000","last_clean":"2024-04-25T15:02:09.562234+0000","last_became_active":"2024-04-25T14:59:45.316604+0000","last_became_peered":"2024-04-25T14:59:45.316604+0000","last_unstale":"2024-04-25T15:02:09.562234+0000","last_undegraded":"2024-04-25T15:02:09.562234+0000","last_fullsized":"2024-04-25T15:02:09.562234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:45.699926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426847+0000","last_change":"2024-04-25T14:59:46.316195+0000","last_active":"2024-04-25T15:02:09.426847+0000","last_peered":"2024-04-25T15:02:09.426847+0000","last_clean":"2024-04-25T15:02:09.426847+0000","last_became_active":"2024-04-25T14:59:46.315809+0000","last_became_peered":"2024-04-25T14:59:46.315809+0000","last_unstale":"2024-04-25T15:02:09.426847+0000","last_undegraded":"2024-04-25T15:02:09.426847+0000","last_fullsized":"2024-04-25T15:02:09.426847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:34.440058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426952+0000","last_change":"2024-04-25T14:59:45.325137+0000","last_active":"2024-04-25T15:02:09.426952+0000","last_peered":"2024-04-25T15:02:09.426952+0000","last_clean":"2024-04-25T15:02:09.426952+0000","last_became_active":"2024-04-25T14:59:45.324769+0000","last_became_peered":"2024-04-25T14:59:45.324769+0000","last_unstale":"2024-04-25T15:02:09.426952+0000","last_undegraded":"2024-04-25T15:02:09.426952+0000","last_fullsized":"2024-04-25T15:02:09.426952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:02.778853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427258+0000","last_change":"2024-04-25T14:59:46.304099+0000","last_active":"2024-04-25T15:02:09.427258+0000","last_peered":"2024-04-25T15:02:09.427258+0000","last_clean":"2024-04-25T15:02:09.427258+0000","last_became_active":"2024-04-25T14:59:46.303774+0000","last_became_peered":"2024-04-25T14:59:46.303774+0000","last_unstale":"2024-04-25T15:02:09.427258+0000","last_undegraded":"2024-04-25T15:02:09.427258+0000","last_fullsized":"2024-04-25T15:02:09.427258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.308212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'65","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:11.551625+0000","last_change":"2024-04-25T14:59:45.322324+0000","last_active":"2024-04-25T15:02:11.551625+0000","last_peered":"2024-04-25T15:02:11.551625+0000","last_clean":"2024-04-25T15:02:11.551625+0000","last_became_active":"2024-04-25T14:59:45.322031+0000","last_became_peered":"2024-04-25T14:59:45.322031+0000","last_unstale":"2024-04-25T15:02:11.551625+0000","last_undegraded":"2024-04-25T15:02:11.551625+0000","last_fullsized":"2024-04-25T15:02:11.551625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.424928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427253+0000","last_change":"2024-04-25T15:00:02.730051+0000","last_active":"2024-04-25T15:02:09.427253+0000","last_peered":"2024-04-25T15:02:09.427253+0000","last_clean":"2024-04-25T15:02:09.427253+0000","last_became_active":"2024-04-25T15:00:02.725610+0000","last_became_peered":"2024-04-25T15:00:02.725610+0000","last_unstale":"2024-04-25T15:02:09.427253+0000","last_undegraded":"2024-04-25T15:02:09.427253+0000","last_fullsized":"2024-04-25T15:02:09.427253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:04.598366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143595+0000","last_change":"2024-04-25T14:59:45.335502+0000","last_active":"2024-04-25T15:02:10.143595+0000","last_peered":"2024-04-25T15:02:10.143595+0000","last_clean":"2024-04-25T15:02:10.143595+0000","last_became_active":"2024-04-25T14:59:45.335018+0000","last_became_peered":"2024-04-25T14:59:45.335018+0000","last_unstale":"2024-04-25T15:02:10.143595+0000","last_undegraded":"2024-04-25T15:02:10.143595+0000","last_fullsized":"2024-04-25T15:02:10.143595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:11.918001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427140+0000","last_change":"2024-04-25T14:59:46.304766+0000","last_active":"2024-04-25T15:02:09.427140+0000","last_peered":"2024-04-25T15:02:09.427140+0000","last_clean":"2024-04-25T15:02:09.427140+0000","last_became_active":"2024-04-25T14:59:46.304413+0000","last_became_peered":"2024-04-25T14:59:46.304413+0000","last_unstale":"2024-04-25T15:02:09.427140+0000","last_undegraded":"2024-04-25T15:02:09.427140+0000","last_fullsized":"2024-04-25T15:02:09.427140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818984+0000","last_change":"2024-04-25T14:59:46.305285+0000","last_active":"2024-04-25T15:02:09.818984+0000","last_peered":"2024-04-25T15:02:09.818984+0000","last_clean":"2024-04-25T15:02:09.818984+0000","last_became_active":"2024-04-25T14:59:46.304934+0000","last_became_peered":"2024-04-25T14:59:46.304934+0000","last_unstale":"2024-04-25T15:02:09.818984+0000","last_undegraded":"2024-04-25T15:02:09.818984+0000","last_fullsized":"2024-04-25T15:02:09.818984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:44.892921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426447+0000","last_change":"2024-04-25T15:00:02.718240+0000","last_active":"2024-04-25T15:02:09.426447+0000","last_peered":"2024-04-25T15:02:09.426447+0000","last_clean":"2024-04-25T15:02:09.426447+0000","last_became_active":"2024-04-25T15:00:02.717792+0000","last_became_peered":"2024-04-25T15:00:02.717792+0000","last_unstale":"2024-04-25T15:02:09.426447+0000","last_undegraded":"2024-04-25T15:02:09.426447+0000","last_fullsized":"2024-04-25T15:02:09.426447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:49.532532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426395+0000","last_change":"2024-04-25T14:59:46.319694+0000","last_active":"2024-04-25T15:02:09.426395+0000","last_peered":"2024-04-25T15:02:09.426395+0000","last_clean":"2024-04-25T15:02:09.426395+0000","last_became_active":"2024-04-25T14:59:46.319417+0000","last_became_peered":"2024-04-25T14:59:46.319417+0000","last_unstale":"2024-04-25T15:02:09.426395+0000","last_undegraded":"2024-04-25T15:02:09.426395+0000","last_fullsized":"2024-04-25T15:02:09.426395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.177498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428062+0000","last_change":"2024-04-25T15:00:02.986849+0000","last_active":"2024-04-25T15:02:09.428062+0000","last_peered":"2024-04-25T15:02:09.428062+0000","last_clean":"2024-04-25T15:02:09.428062+0000","last_became_active":"2024-04-25T15:00:02.986345+0000","last_became_peered":"2024-04-25T15:00:02.986345+0000","last_unstale":"2024-04-25T15:02:09.428062+0000","last_undegraded":"2024-04-25T15:02:09.428062+0000","last_fullsized":"2024-04-25T15:02:09.428062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:22.707469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143409+0000","last_change":"2024-04-25T14:59:46.319167+0000","last_active":"2024-04-25T15:02:10.143409+0000","last_peered":"2024-04-25T15:02:10.143409+0000","last_clean":"2024-04-25T15:02:10.143409+0000","last_became_active":"2024-04-25T14:59:46.318804+0000","last_became_peered":"2024-04-25T14:59:46.318804+0000","last_unstale":"2024-04-25T15:02:10.143409+0000","last_undegraded":"2024-04-25T15:02:10.143409+0000","last_fullsized":"2024-04-25T15:02:10.143409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:34.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818946+0000","last_change":"2024-04-25T14:59:46.304554+0000","last_active":"2024-04-25T15:02:09.818946+0000","last_peered":"2024-04-25T15:02:09.818946+0000","last_clean":"2024-04-25T15:02:09.818946+0000","last_became_active":"2024-04-25T14:59:46.304062+0000","last_became_peered":"2024-04-25T14:59:46.304062+0000","last_unstale":"2024-04-25T15:02:09.818946+0000","last_undegraded":"2024-04-25T15:02:09.818946+0000","last_fullsized":"2024-04-25T15:02:09.818946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426897+0000","last_change":"2024-04-25T14:59:46.299104+0000","last_active":"2024-04-25T15:02:09.426897+0000","last_peered":"2024-04-25T15:02:09.426897+0000","last_clean":"2024-04-25T15:02:09.426897+0000","last_became_active":"2024-04-25T14:59:46.298765+0000","last_became_peered":"2024-04-25T14:59:46.298765+0000","last_unstale":"2024-04-25T15:02:09.426897+0000","last_undegraded":"2024-04-25T15:02:09.426897+0000","last_fullsized":"2024-04-25T15:02:09.426897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.494298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563297+0000","last_change":"2024-04-25T14:59:46.328885+0000","last_active":"2024-04-25T15:02:09.563297+0000","last_peered":"2024-04-25T15:02:09.563297+0000","last_clean":"2024-04-25T15:02:09.563297+0000","last_became_active":"2024-04-25T14:59:46.328375+0000","last_became_peered":"2024-04-25T14:59:46.328375+0000","last_unstale":"2024-04-25T15:02:09.563297+0000","last_undegraded":"2024-04-25T15:02:09.563297+0000","last_fullsized":"2024-04-25T15:02:09.563297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:08.959045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570545+0000","last_change":"2024-04-25T14:59:46.317403+0000","last_active":"2024-04-25T15:02:09.570545+0000","last_peered":"2024-04-25T15:02:09.570545+0000","last_clean":"2024-04-25T15:02:09.570545+0000","last_became_active":"2024-04-25T14:59:46.317100+0000","last_became_peered":"2024-04-25T14:59:46.317100+0000","last_unstale":"2024-04-25T15:02:09.570545+0000","last_undegraded":"2024-04-25T15:02:09.570545+0000","last_fullsized":"2024-04-25T15:02:09.570545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:23.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563030+0000","last_change":"2024-04-25T14:59:46.327147+0000","last_active":"2024-04-25T15:02:09.563030+0000","last_peered":"2024-04-25T15:02:09.563030+0000","last_clean":"2024-04-25T15:02:09.563030+0000","last_became_active":"2024-04-25T14:59:46.321857+0000","last_became_peered":"2024-04-25T14:59:46.321857+0000","last_unstale":"2024-04-25T15:02:09.563030+0000","last_undegraded":"2024-04-25T15:02:09.563030+0000","last_fullsized":"2024-04-25T15:02:09.563030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:46.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740053+0000","last_change":"2024-04-25T14:59:46.310841+0000","last_active":"2024-04-25T15:02:09.740053+0000","last_peered":"2024-04-25T15:02:09.740053+0000","last_clean":"2024-04-25T15:02:09.740053+0000","last_became_active":"2024-04-25T14:59:46.310446+0000","last_became_peered":"2024-04-25T14:59:46.310446+0000","last_unstale":"2024-04-25T15:02:09.740053+0000","last_undegraded":"2024-04-25T15:02:09.740053+0000","last_fullsized":"2024-04-25T15:02:09.740053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.594784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143711+0000","last_change":"2024-04-25T14:59:46.320715+0000","last_active":"2024-04-25T15:02:10.143711+0000","last_peered":"2024-04-25T15:02:10.143711+0000","last_clean":"2024-04-25T15:02:10.143711+0000","last_became_active":"2024-04-25T14:59:46.320238+0000","last_became_peered":"2024-04-25T14:59:46.320238+0000","last_unstale":"2024-04-25T15:02:10.143711+0000","last_undegraded":"2024-04-25T15:02:10.143711+0000","last_fullsized":"2024-04-25T15:02:10.143711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:55.387814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818885+0000","last_change":"2024-04-25T14:59:46.303772+0000","last_active":"2024-04-25T15:02:09.818885+0000","last_peered":"2024-04-25T15:02:09.818885+0000","last_clean":"2024-04-25T15:02:09.818885+0000","last_became_active":"2024-04-25T14:59:46.303435+0000","last_became_peered":"2024-04-25T14:59:46.303435+0000","last_unstale":"2024-04-25T15:02:09.818885+0000","last_undegraded":"2024-04-25T15:02:09.818885+0000","last_fullsized":"2024-04-25T15:02:09.818885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:06.692979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":790006,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":653,"num_read_kb":532,"num_write":322,"num_write_kb":2317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2469888,"data_stored":2370018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351017,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":467709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29080,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716072,"statfs":{"total":95995035648,"available":95965257728,"internally_reserved":0,"allocated":1990656,"data_stored":1454380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36680,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708472,"statfs":{"total":95995035648,"available":95957475328,"internally_reserved":0,"allocated":1372160,"data_stored":861624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28356,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716796,"statfs":{"total":95995035648,"available":95965999104,"internally_reserved":0,"allocated":1372160,"data_stored":861529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1058776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":467826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":535469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":467799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411421,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":535503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28804,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716348,"statfs":{"total":95995035648,"available":95965540352,"internally_reserved":0,"allocated":1708032,"data_stored":1189378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888145,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":467731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724618,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29004,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716148,"statfs":{"total":95995035648,"available":95965335552,"internally_reserved":0,"allocated":1912832,"data_stored":1386608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":733184,"data_stored":721552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":328414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":396186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:02:19.622 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T15:02:19.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 73 op/s 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: Updating smithi137:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: Updating smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: Updating smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:02:19.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:19.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:19 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.124 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:02:20.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:20.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:20.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.475 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:02:20.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:20.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:20.476 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:20 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:02:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:20.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:20.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:20 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: Updating smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:02:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='client.25228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:02:20.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:02:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:02:20.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:20 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:02:21.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:21 smithi094 ceph-mon[27862]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 57 op/s 2024-04-25T15:02:21.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:21 smithi137 ceph-mon[32439]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 57 op/s 2024-04-25T15:02:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:21 smithi138 ceph-mon[32875]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 102 KiB/s wr, 57 op/s 2024-04-25T15:02:22.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 -- 172.21.15.94:0/3568072909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c075590 msgr2=0x7fd70c10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:22.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 --2- 172.21.15.94:0/3568072909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c075590 0x7fd70c10ef00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8009990 tx=0x7fd6f8009ca0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 -- 172.21.15.94:0/3568072909 shutdown_connections 2024-04-25T15:02:22.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 --2- 172.21.15.94:0/3568072909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10f440 0x7fd70c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 --2- 172.21.15.94:0/3568072909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 0x7fd70c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 --2- 172.21.15.94:0/3568072909 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c075590 0x7fd70c10ef00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 -- 172.21.15.94:0/3568072909 >> 172.21.15.94:0/3568072909 conn(0x7fd70c079ba0 msgr2=0x7fd70c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:22.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 -- 172.21.15.94:0/3568072909 shutdown_connections 2024-04-25T15:02:22.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.211+0000 7fd712e00700 1 -- 172.21.15.94:0/3568072909 wait complete. 2024-04-25T15:02:22.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.212+0000 7fd712e00700 1 Processor -- start 2024-04-25T15:02:22.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.213+0000 7fd712e00700 1 -- start start 2024-04-25T15:02:22.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.213+0000 7fd712e00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 0x7fd70c19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:22.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.213+0000 7fd712e00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c075590 0x7fd70c19b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:22.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.213+0000 7fd712e00700 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c10f440 0x7fd70c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.213+0000 7fd712e00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c114e70 con 0x7fd70c074c70 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.213+0000 7fd712e00700 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c114cf0 con 0x7fd70c10f440 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.213+0000 7fd712e00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c114ff0 con 0x7fd70c075590 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 0x7fd70c19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 0x7fd70c19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52342/0 (socket says 172.21.15.94:52342) 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 -- 172.21.15.94:0/2142959307 learned_addr learned my addr 172.21.15.94:0/2142959307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70b400700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c075590 0x7fd70c19b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd710e00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c10f440 0x7fd70c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 -- 172.21.15.94:0/2142959307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c075590 msgr2=0x7fd70c19b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c075590 0x7fd70c19b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 -- 172.21.15.94:0/2142959307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c10f440 msgr2=0x7fd70c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c10f440 0x7fd70c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.214+0000 7fd70be00700 1 -- 172.21.15.94:0/2142959307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c1a0300 con 0x7fd70c074c70 2024-04-25T15:02:22.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.215+0000 7fd70be00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 0x7fd70c19b340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00e9a0 tx=0x7fd6fc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:22.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.215+0000 7fd708c00700 1 -- 172.21.15.94:0/2142959307 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00cbd0 con 0x7fd70c074c70 2024-04-25T15:02:22.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.215+0000 7fd708c00700 1 -- 172.21.15.94:0/2142959307 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6fc00cd70 con 0x7fd70c074c70 2024-04-25T15:02:22.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.215+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c1a05f0 con 0x7fd70c074c70 2024-04-25T15:02:22.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.215+0000 7fd708c00700 1 -- 172.21.15.94:0/2142959307 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc018680 con 0x7fd70c074c70 2024-04-25T15:02:22.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.215+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c1a7e70 con 0x7fd70c074c70 2024-04-25T15:02:22.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.216+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c069730 con 0x7fd70c074c70 2024-04-25T15:02:22.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.218+0000 7fd708c00700 1 -- 172.21.15.94:0/2142959307 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fd6fc010430 con 0x7fd70c074c70 2024-04-25T15:02:22.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.220+0000 7fd708c00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd6f00a3ec0 0x7fd6f00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:22.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.220+0000 7fd708c00700 1 -- 172.21.15.94:0/2142959307 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fd6fc0d0ba0 con 0x7fd70c074c70 2024-04-25T15:02:22.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.220+0000 7fd70b400700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd6f00a3ec0 0x7fd6f00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:22.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.221+0000 7fd70b400700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd6f00a3ec0 0x7fd6f00a6380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd70c111100 tx=0x7fd6f8005fd0 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:22.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.225+0000 7fd708c00700 1 -- 172.21.15.94:0/2142959307 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6fc09d080 con 0x7fd70c074c70 2024-04-25T15:02:22.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.445+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd70c062e80 con 0x7fd6f00a3ec0 2024-04-25T15:02:22.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.454+0000 7fd708c00700 1 -- 172.21.15.94:0/2142959307 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257688 (secure 0 0 0) 0x7fd6f8066eb0 con 0x7fd6f00a3ec0 2024-04-25T15:02:22.460 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd6f00a3ec0 msgr2=0x7fd6f00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd6f00a3ec0 0x7fd6f00a6380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd70c111100 tx=0x7fd6f8005fd0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 msgr2=0x7fd70c19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 0x7fd70c19b340 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00e9a0 tx=0x7fd6fc00ecb0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 shutdown_connections 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd6f00a3ec0 0x7fd6f00a6380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd70c10f440 0x7fd70c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd70c074c70 0x7fd70c19b340 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 --2- 172.21.15.94:0/2142959307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c075590 0x7fd70c19b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:22.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 >> 172.21.15.94:0/2142959307 conn(0x7fd70c079ba0 msgr2=0x7fd70c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:22.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 shutdown_connections 2024-04-25T15:02:22.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:22.460+0000 7fd712e00700 1 -- 172.21.15.94:0/2142959307 wait complete. 2024-04-25T15:02:22.463 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-04-25T15:02:23.206 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-04-25T15:02:21.451968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1380608,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":900,"num_read_kb":867,"num_write":525,"num_write_kb":5257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180853,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381732,"kb_used_data":15552,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560092,"statfs":{"total":1151940427776,"available":1151549534208,"internally_reserved":0,"allocated":15925248,"data_stored":9754332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":374,"num_read_kb":451,"num_write":148,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.982608"},"pg_stats":[{"pgid":"3.3f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.819097+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:02:09.819097+0000","last_peered":"2024-04-25T15:02:09.819097+0000","last_clean":"2024-04-25T15:02:09.819097+0000","last_became_active":"2024-04-25T14:59:46.311856+0000","last_became_peered":"2024-04-25T14:59:46.311856+0000","last_unstale":"2024-04-25T15:02:09.819097+0000","last_undegraded":"2024-04-25T15:02:09.819097+0000","last_fullsized":"2024-04-25T15:02:09.819097+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:07.469645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427851+0000","last_change":"2024-04-25T14:59:46.310292+0000","last_active":"2024-04-25T15:02:09.427851+0000","last_peered":"2024-04-25T15:02:09.427851+0000","last_clean":"2024-04-25T15:02:09.427851+0000","last_became_active":"2024-04-25T14:59:46.309812+0000","last_became_peered":"2024-04-25T14:59:46.309812+0000","last_unstale":"2024-04-25T15:02:09.427851+0000","last_undegraded":"2024-04-25T15:02:09.427851+0000","last_fullsized":"2024-04-25T15:02:09.427851+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:52.448406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428156+0000","last_change":"2024-04-25T14:59:46.300987+0000","last_active":"2024-04-25T15:02:09.428156+0000","last_peered":"2024-04-25T15:02:09.428156+0000","last_clean":"2024-04-25T15:02:09.428156+0000","last_became_active":"2024-04-25T14:59:46.299866+0000","last_became_peered":"2024-04-25T14:59:46.299866+0000","last_unstale":"2024-04-25T15:02:09.428156+0000","last_undegraded":"2024-04-25T15:02:09.428156+0000","last_fullsized":"2024-04-25T15:02:09.428156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:09.831131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457562+0000","last_change":"2024-04-25T14:59:46.304278+0000","last_active":"2024-04-25T15:02:09.457562+0000","last_peered":"2024-04-25T15:02:09.457562+0000","last_clean":"2024-04-25T15:02:09.457562+0000","last_became_active":"2024-04-25T14:59:46.303956+0000","last_became_peered":"2024-04-25T14:59:46.303956+0000","last_unstale":"2024-04-25T15:02:09.457562+0000","last_undegraded":"2024-04-25T15:02:09.457562+0000","last_fullsized":"2024-04-25T15:02:09.457562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:31.752548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562357+0000","last_change":"2024-04-25T14:59:46.323730+0000","last_active":"2024-04-25T15:02:09.562357+0000","last_peered":"2024-04-25T15:02:09.562357+0000","last_clean":"2024-04-25T15:02:09.562357+0000","last_became_active":"2024-04-25T14:59:46.323383+0000","last_became_peered":"2024-04-25T14:59:46.323383+0000","last_unstale":"2024-04-25T15:02:09.562357+0000","last_undegraded":"2024-04-25T15:02:09.562357+0000","last_fullsized":"2024-04-25T15:02:09.562357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:50.846392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574191+0000","last_change":"2024-04-25T14:59:46.317623+0000","last_active":"2024-04-25T15:02:09.574191+0000","last_peered":"2024-04-25T15:02:09.574191+0000","last_clean":"2024-04-25T15:02:09.574191+0000","last_became_active":"2024-04-25T14:59:46.317255+0000","last_became_peered":"2024-04-25T14:59:46.317255+0000","last_unstale":"2024-04-25T15:02:09.574191+0000","last_undegraded":"2024-04-25T15:02:09.574191+0000","last_fullsized":"2024-04-25T15:02:09.574191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:24.449615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426705+0000","last_change":"2024-04-25T15:00:02.723289+0000","last_active":"2024-04-25T15:02:09.426705+0000","last_peered":"2024-04-25T15:02:09.426705+0000","last_clean":"2024-04-25T15:02:09.426705+0000","last_became_active":"2024-04-25T15:00:02.722968+0000","last_became_peered":"2024-04-25T15:00:02.722968+0000","last_unstale":"2024-04-25T15:02:09.426705+0000","last_undegraded":"2024-04-25T15:02:09.426705+0000","last_fullsized":"2024-04-25T15:02:09.426705+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:45.674905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562332+0000","last_change":"2024-04-25T14:59:46.327687+0000","last_active":"2024-04-25T15:02:09.562332+0000","last_peered":"2024-04-25T15:02:09.562332+0000","last_clean":"2024-04-25T15:02:09.562332+0000","last_became_active":"2024-04-25T14:59:46.325905+0000","last_became_peered":"2024-04-25T14:59:46.325905+0000","last_unstale":"2024-04-25T15:02:09.562332+0000","last_undegraded":"2024-04-25T15:02:09.562332+0000","last_fullsized":"2024-04-25T15:02:09.562332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:50.058801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740112+0000","last_change":"2024-04-25T14:59:46.304722+0000","last_active":"2024-04-25T15:02:09.740112+0000","last_peered":"2024-04-25T15:02:09.740112+0000","last_clean":"2024-04-25T15:02:09.740112+0000","last_became_active":"2024-04-25T14:59:46.304391+0000","last_became_peered":"2024-04-25T14:59:46.304391+0000","last_unstale":"2024-04-25T15:02:09.740112+0000","last_undegraded":"2024-04-25T15:02:09.740112+0000","last_fullsized":"2024-04-25T15:02:09.740112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:37:15.735670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574449+0000","last_change":"2024-04-25T14:59:46.307796+0000","last_active":"2024-04-25T15:02:09.574449+0000","last_peered":"2024-04-25T15:02:09.574449+0000","last_clean":"2024-04-25T15:02:09.574449+0000","last_became_active":"2024-04-25T14:59:46.306741+0000","last_became_peered":"2024-04-25T14:59:46.306741+0000","last_unstale":"2024-04-25T15:02:09.574449+0000","last_undegraded":"2024-04-25T15:02:09.574449+0000","last_fullsized":"2024-04-25T15:02:09.574449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:34.407272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427124+0000","last_change":"2024-04-25T14:59:46.306390+0000","last_active":"2024-04-25T15:02:09.427124+0000","last_peered":"2024-04-25T15:02:09.427124+0000","last_clean":"2024-04-25T15:02:09.427124+0000","last_became_active":"2024-04-25T14:59:46.306026+0000","last_became_peered":"2024-04-25T14:59:46.306026+0000","last_unstale":"2024-04-25T15:02:09.427124+0000","last_undegraded":"2024-04-25T15:02:09.427124+0000","last_fullsized":"2024-04-25T15:02:09.427124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:59.340382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457480+0000","last_change":"2024-04-25T14:59:46.305840+0000","last_active":"2024-04-25T15:02:09.457480+0000","last_peered":"2024-04-25T15:02:09.457480+0000","last_clean":"2024-04-25T15:02:09.457480+0000","last_became_active":"2024-04-25T14:59:46.304867+0000","last_became_peered":"2024-04-25T14:59:46.304867+0000","last_unstale":"2024-04-25T15:02:09.457480+0000","last_undegraded":"2024-04-25T15:02:09.457480+0000","last_fullsized":"2024-04-25T15:02:09.457480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:58.569011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143755+0000","last_change":"2024-04-25T14:59:46.318044+0000","last_active":"2024-04-25T15:02:10.143755+0000","last_peered":"2024-04-25T15:02:10.143755+0000","last_clean":"2024-04-25T15:02:10.143755+0000","last_became_active":"2024-04-25T14:59:46.315935+0000","last_became_peered":"2024-04-25T14:59:46.315935+0000","last_unstale":"2024-04-25T15:02:10.143755+0000","last_undegraded":"2024-04-25T15:02:10.143755+0000","last_fullsized":"2024-04-25T15:02:10.143755+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:42.366802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427570+0000","last_change":"2024-04-25T14:59:46.308565+0000","last_active":"2024-04-25T15:02:09.427570+0000","last_peered":"2024-04-25T15:02:09.427570+0000","last_clean":"2024-04-25T15:02:09.427570+0000","last_became_active":"2024-04-25T14:59:46.306198+0000","last_became_peered":"2024-04-25T14:59:46.306198+0000","last_unstale":"2024-04-25T15:02:09.427570+0000","last_undegraded":"2024-04-25T15:02:09.427570+0000","last_fullsized":"2024-04-25T15:02:09.427570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:12.686162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427475+0000","last_change":"2024-04-25T14:59:46.308313+0000","last_active":"2024-04-25T15:02:09.427475+0000","last_peered":"2024-04-25T15:02:09.427475+0000","last_clean":"2024-04-25T15:02:09.427475+0000","last_became_active":"2024-04-25T14:59:46.305974+0000","last_became_peered":"2024-04-25T14:59:46.305974+0000","last_unstale":"2024-04-25T15:02:09.427475+0000","last_undegraded":"2024-04-25T15:02:09.427475+0000","last_fullsized":"2024-04-25T15:02:09.427475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:42.841983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.739661+0000","last_change":"2024-04-25T14:59:46.302726+0000","last_active":"2024-04-25T15:02:09.739661+0000","last_peered":"2024-04-25T15:02:09.739661+0000","last_clean":"2024-04-25T15:02:09.739661+0000","last_became_active":"2024-04-25T14:59:46.302406+0000","last_became_peered":"2024-04-25T14:59:46.302406+0000","last_unstale":"2024-04-25T15:02:09.739661+0000","last_undegraded":"2024-04-25T15:02:09.739661+0000","last_fullsized":"2024-04-25T15:02:09.739661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:02.073683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.819221+0000","last_change":"2024-04-25T14:59:46.312174+0000","last_active":"2024-04-25T15:02:09.819221+0000","last_peered":"2024-04-25T15:02:09.819221+0000","last_clean":"2024-04-25T15:02:09.819221+0000","last_became_active":"2024-04-25T14:59:46.311822+0000","last_became_peered":"2024-04-25T14:59:46.311822+0000","last_unstale":"2024-04-25T15:02:09.819221+0000","last_undegraded":"2024-04-25T15:02:09.819221+0000","last_fullsized":"2024-04-25T15:02:09.819221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:45.128537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559804+0000","last_change":"2024-04-25T14:59:46.313816+0000","last_active":"2024-04-25T15:02:09.559804+0000","last_peered":"2024-04-25T15:02:09.559804+0000","last_clean":"2024-04-25T15:02:09.559804+0000","last_became_active":"2024-04-25T14:59:46.313462+0000","last_became_peered":"2024-04-25T14:59:46.313462+0000","last_unstale":"2024-04-25T15:02:09.559804+0000","last_undegraded":"2024-04-25T15:02:09.559804+0000","last_fullsized":"2024-04-25T15:02:09.559804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:29.402113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426726+0000","last_change":"2024-04-25T14:59:46.317972+0000","last_active":"2024-04-25T15:02:09.426726+0000","last_peered":"2024-04-25T15:02:09.426726+0000","last_clean":"2024-04-25T15:02:09.426726+0000","last_became_active":"2024-04-25T14:59:46.317466+0000","last_became_peered":"2024-04-25T14:59:46.317466+0000","last_unstale":"2024-04-25T15:02:09.426726+0000","last_undegraded":"2024-04-25T15:02:09.426726+0000","last_fullsized":"2024-04-25T15:02:09.426726+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:03.706663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559858+0000","last_change":"2024-04-25T14:59:45.320752+0000","last_active":"2024-04-25T15:02:09.559858+0000","last_peered":"2024-04-25T15:02:09.559858+0000","last_clean":"2024-04-25T15:02:09.559858+0000","last_became_active":"2024-04-25T14:59:45.319463+0000","last_became_peered":"2024-04-25T14:59:45.319463+0000","last_unstale":"2024-04-25T15:02:09.559858+0000","last_undegraded":"2024-04-25T15:02:09.559858+0000","last_fullsized":"2024-04-25T15:02:09.559858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:52.608899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563574+0000","last_change":"2024-04-25T14:59:46.323802+0000","last_active":"2024-04-25T15:02:09.563574+0000","last_peered":"2024-04-25T15:02:09.563574+0000","last_clean":"2024-04-25T15:02:09.563574+0000","last_became_active":"2024-04-25T14:59:46.323498+0000","last_became_peered":"2024-04-25T14:59:46.323498+0000","last_unstale":"2024-04-25T15:02:09.563574+0000","last_undegraded":"2024-04-25T15:02:09.563574+0000","last_fullsized":"2024-04-25T15:02:09.563574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:38.236936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563729+0000","last_change":"2024-04-25T14:59:45.335156+0000","last_active":"2024-04-25T15:02:09.563729+0000","last_peered":"2024-04-25T15:02:09.563729+0000","last_clean":"2024-04-25T15:02:09.563729+0000","last_became_active":"2024-04-25T14:59:45.333589+0000","last_became_peered":"2024-04-25T14:59:45.333589+0000","last_unstale":"2024-04-25T15:02:09.563729+0000","last_undegraded":"2024-04-25T15:02:09.563729+0000","last_fullsized":"2024-04-25T15:02:09.563729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.759504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428190+0000","last_change":"2024-04-25T14:59:46.319165+0000","last_active":"2024-04-25T15:02:09.428190+0000","last_peered":"2024-04-25T15:02:09.428190+0000","last_clean":"2024-04-25T15:02:09.428190+0000","last_became_active":"2024-04-25T14:59:46.318801+0000","last_became_peered":"2024-04-25T14:59:46.318801+0000","last_unstale":"2024-04-25T15:02:09.428190+0000","last_undegraded":"2024-04-25T15:02:09.428190+0000","last_fullsized":"2024-04-25T15:02:09.428190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:01.806099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562599+0000","last_change":"2024-04-25T14:59:45.317956+0000","last_active":"2024-04-25T15:02:09.562599+0000","last_peered":"2024-04-25T15:02:09.562599+0000","last_clean":"2024-04-25T15:02:09.562599+0000","last_became_active":"2024-04-25T14:59:45.317655+0000","last_became_peered":"2024-04-25T14:59:45.317655+0000","last_unstale":"2024-04-25T15:02:09.562599+0000","last_undegraded":"2024-04-25T15:02:09.562599+0000","last_fullsized":"2024-04-25T15:02:09.562599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:59.008411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740141+0000","last_change":"2024-04-25T14:59:46.311659+0000","last_active":"2024-04-25T15:02:09.740141+0000","last_peered":"2024-04-25T15:02:09.740141+0000","last_clean":"2024-04-25T15:02:09.740141+0000","last_became_active":"2024-04-25T14:59:46.311287+0000","last_became_peered":"2024-04-25T14:59:46.311287+0000","last_unstale":"2024-04-25T15:02:09.740141+0000","last_undegraded":"2024-04-25T15:02:09.740141+0000","last_fullsized":"2024-04-25T15:02:09.740141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:27:29.434223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562463+0000","last_change":"2024-04-25T14:59:45.317658+0000","last_active":"2024-04-25T15:02:09.562463+0000","last_peered":"2024-04-25T15:02:09.562463+0000","last_clean":"2024-04-25T15:02:09.562463+0000","last_became_active":"2024-04-25T14:59:45.317311+0000","last_became_peered":"2024-04-25T14:59:45.317311+0000","last_unstale":"2024-04-25T15:02:09.562463+0000","last_undegraded":"2024-04-25T15:02:09.562463+0000","last_fullsized":"2024-04-25T15:02:09.562463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:05.602412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570817+0000","last_change":"2024-04-25T14:59:46.305456+0000","last_active":"2024-04-25T15:02:09.570817+0000","last_peered":"2024-04-25T15:02:09.570817+0000","last_clean":"2024-04-25T15:02:09.570817+0000","last_became_active":"2024-04-25T14:59:46.305143+0000","last_became_peered":"2024-04-25T14:59:46.305143+0000","last_unstale":"2024-04-25T15:02:09.570817+0000","last_undegraded":"2024-04-25T15:02:09.570817+0000","last_fullsized":"2024-04-25T15:02:09.570817+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:03.941200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570635+0000","last_change":"2024-04-25T14:59:45.315407+0000","last_active":"2024-04-25T15:02:09.570635+0000","last_peered":"2024-04-25T15:02:09.570635+0000","last_clean":"2024-04-25T15:02:09.570635+0000","last_became_active":"2024-04-25T14:59:45.315070+0000","last_became_peered":"2024-04-25T14:59:45.315070+0000","last_unstale":"2024-04-25T15:02:09.570635+0000","last_undegraded":"2024-04-25T15:02:09.570635+0000","last_fullsized":"2024-04-25T15:02:09.570635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:32.492027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562302+0000","last_change":"2024-04-25T14:59:46.318218+0000","last_active":"2024-04-25T15:02:09.562302+0000","last_peered":"2024-04-25T15:02:09.562302+0000","last_clean":"2024-04-25T15:02:09.562302+0000","last_became_active":"2024-04-25T14:59:46.317855+0000","last_became_peered":"2024-04-25T14:59:46.317855+0000","last_unstale":"2024-04-25T15:02:09.562302+0000","last_undegraded":"2024-04-25T15:02:09.562302+0000","last_fullsized":"2024-04-25T15:02:09.562302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:11.712691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574151+0000","last_change":"2024-04-25T14:59:45.319624+0000","last_active":"2024-04-25T15:02:09.574151+0000","last_peered":"2024-04-25T15:02:09.574151+0000","last_clean":"2024-04-25T15:02:09.574151+0000","last_became_active":"2024-04-25T14:59:45.319310+0000","last_became_peered":"2024-04-25T14:59:45.319310+0000","last_unstale":"2024-04-25T15:02:09.574151+0000","last_undegraded":"2024-04-25T15:02:09.574151+0000","last_fullsized":"2024-04-25T15:02:09.574151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:40.741166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570715+0000","last_change":"2024-04-25T14:59:46.317382+0000","last_active":"2024-04-25T15:02:09.570715+0000","last_peered":"2024-04-25T15:02:09.570715+0000","last_clean":"2024-04-25T15:02:09.570715+0000","last_became_active":"2024-04-25T14:59:46.316993+0000","last_became_peered":"2024-04-25T14:59:46.316993+0000","last_unstale":"2024-04-25T15:02:09.570715+0000","last_undegraded":"2024-04-25T15:02:09.570715+0000","last_fullsized":"2024-04-25T15:02:09.570715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:06.618964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143699+0000","last_change":"2024-04-25T14:59:45.322731+0000","last_active":"2024-04-25T15:02:10.143699+0000","last_peered":"2024-04-25T15:02:10.143699+0000","last_clean":"2024-04-25T15:02:10.143699+0000","last_became_active":"2024-04-25T14:59:45.322360+0000","last_became_peered":"2024-04-25T14:59:45.322360+0000","last_unstale":"2024-04-25T15:02:10.143699+0000","last_undegraded":"2024-04-25T15:02:10.143699+0000","last_fullsized":"2024-04-25T15:02:10.143699+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:02.890016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562422+0000","last_change":"2024-04-25T14:59:46.327195+0000","last_active":"2024-04-25T15:02:09.562422+0000","last_peered":"2024-04-25T15:02:09.562422+0000","last_clean":"2024-04-25T15:02:09.562422+0000","last_became_active":"2024-04-25T14:59:46.322053+0000","last_became_peered":"2024-04-25T14:59:46.322053+0000","last_unstale":"2024-04-25T15:02:09.562422+0000","last_undegraded":"2024-04-25T15:02:09.562422+0000","last_fullsized":"2024-04-25T15:02:09.562422+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:35.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570537+0000","last_change":"2024-04-25T14:59:45.315688+0000","last_active":"2024-04-25T15:02:09.570537+0000","last_peered":"2024-04-25T15:02:09.570537+0000","last_clean":"2024-04-25T15:02:09.570537+0000","last_became_active":"2024-04-25T14:59:45.315074+0000","last_became_peered":"2024-04-25T14:59:45.315074+0000","last_unstale":"2024-04-25T15:02:09.570537+0000","last_undegraded":"2024-04-25T15:02:09.570537+0000","last_fullsized":"2024-04-25T15:02:09.570537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:49.039794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559531+0000","last_change":"2024-04-25T14:59:46.322874+0000","last_active":"2024-04-25T15:02:09.559531+0000","last_peered":"2024-04-25T15:02:09.559531+0000","last_clean":"2024-04-25T15:02:09.559531+0000","last_became_active":"2024-04-25T14:59:46.322492+0000","last_became_peered":"2024-04-25T14:59:46.322492+0000","last_unstale":"2024-04-25T15:02:09.559531+0000","last_undegraded":"2024-04-25T15:02:09.559531+0000","last_fullsized":"2024-04-25T15:02:09.559531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:21.294792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574373+0000","last_change":"2024-04-25T14:59:45.322448+0000","last_active":"2024-04-25T15:02:09.574373+0000","last_peered":"2024-04-25T15:02:09.574373+0000","last_clean":"2024-04-25T15:02:09.574373+0000","last_became_active":"2024-04-25T14:59:45.322127+0000","last_became_peered":"2024-04-25T14:59:45.322127+0000","last_unstale":"2024-04-25T15:02:09.574373+0000","last_undegraded":"2024-04-25T15:02:09.574373+0000","last_fullsized":"2024-04-25T15:02:09.574373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:31.284668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740025+0000","last_change":"2024-04-25T14:59:46.318371+0000","last_active":"2024-04-25T15:02:09.740025+0000","last_peered":"2024-04-25T15:02:09.740025+0000","last_clean":"2024-04-25T15:02:09.740025+0000","last_became_active":"2024-04-25T14:59:46.317819+0000","last_became_peered":"2024-04-25T14:59:46.317819+0000","last_unstale":"2024-04-25T15:02:09.740025+0000","last_undegraded":"2024-04-25T15:02:09.740025+0000","last_fullsized":"2024-04-25T15:02:09.740025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.047982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427082+0000","last_change":"2024-04-25T14:59:45.318286+0000","last_active":"2024-04-25T15:02:09.427082+0000","last_peered":"2024-04-25T15:02:09.427082+0000","last_clean":"2024-04-25T15:02:09.427082+0000","last_became_active":"2024-04-25T14:59:45.317787+0000","last_became_peered":"2024-04-25T14:59:45.317787+0000","last_unstale":"2024-04-25T15:02:09.427082+0000","last_undegraded":"2024-04-25T15:02:09.427082+0000","last_fullsized":"2024-04-25T15:02:09.427082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:06.508164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428089+0000","last_change":"2024-04-25T15:00:02.988627+0000","last_active":"2024-04-25T15:02:09.428089+0000","last_peered":"2024-04-25T15:02:09.428089+0000","last_clean":"2024-04-25T15:02:09.428089+0000","last_became_active":"2024-04-25T15:00:02.987379+0000","last_became_peered":"2024-04-25T15:00:02.987379+0000","last_unstale":"2024-04-25T15:02:09.428089+0000","last_undegraded":"2024-04-25T15:02:09.428089+0000","last_fullsized":"2024-04-25T15:02:09.428089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:38.564177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570880+0000","last_change":"2024-04-25T14:59:45.327951+0000","last_active":"2024-04-25T15:02:09.570880+0000","last_peered":"2024-04-25T15:02:09.570880+0000","last_clean":"2024-04-25T15:02:09.570880+0000","last_became_active":"2024-04-25T14:59:45.327660+0000","last_became_peered":"2024-04-25T14:59:45.327660+0000","last_unstale":"2024-04-25T15:02:09.570880+0000","last_undegraded":"2024-04-25T15:02:09.570880+0000","last_fullsized":"2024-04-25T15:02:09.570880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:29.862798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740151+0000","last_change":"2024-04-25T14:59:46.304038+0000","last_active":"2024-04-25T15:02:09.740151+0000","last_peered":"2024-04-25T15:02:09.740151+0000","last_clean":"2024-04-25T15:02:09.740151+0000","last_became_active":"2024-04-25T14:59:46.303713+0000","last_became_peered":"2024-04-25T14:59:46.303713+0000","last_unstale":"2024-04-25T15:02:09.740151+0000","last_undegraded":"2024-04-25T15:02:09.740151+0000","last_fullsized":"2024-04-25T15:02:09.740151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:30.476591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'33","reported_seq":367,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:19.964643+0000","last_change":"2024-04-25T14:59:45.315738+0000","last_active":"2024-04-25T15:02:19.964643+0000","last_peered":"2024-04-25T15:02:19.964643+0000","last_clean":"2024-04-25T15:02:19.964643+0000","last_became_active":"2024-04-25T14:59:45.315423+0000","last_became_peered":"2024-04-25T14:59:45.315423+0000","last_unstale":"2024-04-25T15:02:19.964643+0000","last_undegraded":"2024-04-25T15:02:19.964643+0000","last_fullsized":"2024-04-25T15:02:19.964643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:33.135348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67760,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":312,"num_read_kb":222,"num_write":33,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562433+0000","last_change":"2024-04-25T14:59:46.308085+0000","last_active":"2024-04-25T15:02:09.562433+0000","last_peered":"2024-04-25T15:02:09.562433+0000","last_clean":"2024-04-25T15:02:09.562433+0000","last_became_active":"2024-04-25T14:59:46.307739+0000","last_became_peered":"2024-04-25T14:59:46.307739+0000","last_unstale":"2024-04-25T15:02:09.562433+0000","last_undegraded":"2024-04-25T15:02:09.562433+0000","last_fullsized":"2024-04-25T15:02:09.562433+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:57.949147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426603+0000","last_change":"2024-04-25T14:59:45.327058+0000","last_active":"2024-04-25T15:02:09.426603+0000","last_peered":"2024-04-25T15:02:09.426603+0000","last_clean":"2024-04-25T15:02:09.426603+0000","last_became_active":"2024-04-25T14:59:45.326817+0000","last_became_peered":"2024-04-25T14:59:45.326817+0000","last_unstale":"2024-04-25T15:02:09.426603+0000","last_undegraded":"2024-04-25T15:02:09.426603+0000","last_fullsized":"2024-04-25T15:02:09.426603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:37:51.087966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143375+0000","last_change":"2024-04-25T14:59:46.328901+0000","last_active":"2024-04-25T15:02:10.143375+0000","last_peered":"2024-04-25T15:02:10.143375+0000","last_clean":"2024-04-25T15:02:10.143375+0000","last_became_active":"2024-04-25T14:59:46.328496+0000","last_became_peered":"2024-04-25T14:59:46.328496+0000","last_unstale":"2024-04-25T15:02:10.143375+0000","last_undegraded":"2024-04-25T15:02:10.143375+0000","last_fullsized":"2024-04-25T15:02:10.143375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:48.149719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570566+0000","last_change":"2024-04-25T14:59:45.316673+0000","last_active":"2024-04-25T15:02:09.570566+0000","last_peered":"2024-04-25T15:02:09.570566+0000","last_clean":"2024-04-25T15:02:09.570566+0000","last_became_active":"2024-04-25T14:59:45.316280+0000","last_became_peered":"2024-04-25T14:59:45.316280+0000","last_unstale":"2024-04-25T15:02:09.570566+0000","last_undegraded":"2024-04-25T15:02:09.570566+0000","last_fullsized":"2024-04-25T15:02:09.570566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:51.288236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427119+0000","last_change":"2024-04-25T14:59:46.323821+0000","last_active":"2024-04-25T15:02:09.427119+0000","last_peered":"2024-04-25T15:02:09.427119+0000","last_clean":"2024-04-25T15:02:09.427119+0000","last_became_active":"2024-04-25T14:59:46.323434+0000","last_became_peered":"2024-04-25T14:59:46.323434+0000","last_unstale":"2024-04-25T15:02:09.427119+0000","last_undegraded":"2024-04-25T15:02:09.427119+0000","last_fullsized":"2024-04-25T15:02:09.427119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:20.983270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457421+0000","last_change":"2024-04-25T14:59:45.317953+0000","last_active":"2024-04-25T15:02:09.457421+0000","last_peered":"2024-04-25T15:02:09.457421+0000","last_clean":"2024-04-25T15:02:09.457421+0000","last_became_active":"2024-04-25T14:59:45.317617+0000","last_became_peered":"2024-04-25T14:59:45.317617+0000","last_unstale":"2024-04-25T15:02:09.457421+0000","last_undegraded":"2024-04-25T15:02:09.457421+0000","last_fullsized":"2024-04-25T15:02:09.457421+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:44.235492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426566+0000","last_change":"2024-04-25T14:59:46.305929+0000","last_active":"2024-04-25T15:02:09.426566+0000","last_peered":"2024-04-25T15:02:09.426566+0000","last_clean":"2024-04-25T15:02:09.426566+0000","last_became_active":"2024-04-25T14:59:46.305604+0000","last_became_peered":"2024-04-25T14:59:46.305604+0000","last_unstale":"2024-04-25T15:02:09.426566+0000","last_undegraded":"2024-04-25T15:02:09.426566+0000","last_fullsized":"2024-04-25T15:02:09.426566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:37.340009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818828+0000","last_change":"2024-04-25T15:00:02.731349+0000","last_active":"2024-04-25T15:02:09.818828+0000","last_peered":"2024-04-25T15:02:09.818828+0000","last_clean":"2024-04-25T15:02:09.818828+0000","last_became_active":"2024-04-25T15:00:02.723113+0000","last_became_peered":"2024-04-25T15:00:02.723113+0000","last_unstale":"2024-04-25T15:02:09.818828+0000","last_undegraded":"2024-04-25T15:02:09.818828+0000","last_fullsized":"2024-04-25T15:02:09.818828+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:35.964461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559688+0000","last_change":"2024-04-25T15:00:03.077923+0000","last_active":"2024-04-25T15:02:09.559688+0000","last_peered":"2024-04-25T15:02:09.559688+0000","last_clean":"2024-04-25T15:02:09.559688+0000","last_became_active":"2024-04-25T15:00:03.076709+0000","last_became_peered":"2024-04-25T15:00:03.076709+0000","last_unstale":"2024-04-25T15:02:09.559688+0000","last_undegraded":"2024-04-25T15:02:09.559688+0000","last_fullsized":"2024-04-25T15:02:09.559688+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:22.427477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562656+0000","last_change":"2024-04-25T14:59:45.318291+0000","last_active":"2024-04-25T15:02:09.562656+0000","last_peered":"2024-04-25T15:02:09.562656+0000","last_clean":"2024-04-25T15:02:09.562656+0000","last_became_active":"2024-04-25T14:59:45.317590+0000","last_became_peered":"2024-04-25T14:59:45.317590+0000","last_unstale":"2024-04-25T15:02:09.562656+0000","last_undegraded":"2024-04-25T15:02:09.562656+0000","last_fullsized":"2024-04-25T15:02:09.562656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:11.324000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818949+0000","last_change":"2024-04-25T14:59:46.303503+0000","last_active":"2024-04-25T15:02:09.818949+0000","last_peered":"2024-04-25T15:02:09.818949+0000","last_clean":"2024-04-25T15:02:09.818949+0000","last_became_active":"2024-04-25T14:59:46.303127+0000","last_became_peered":"2024-04-25T14:59:46.303127+0000","last_unstale":"2024-04-25T15:02:09.818949+0000","last_undegraded":"2024-04-25T15:02:09.818949+0000","last_fullsized":"2024-04-25T15:02:09.818949+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:46.327398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559547+0000","last_change":"2024-04-25T14:59:45.330010+0000","last_active":"2024-04-25T15:02:09.559547+0000","last_peered":"2024-04-25T15:02:09.559547+0000","last_clean":"2024-04-25T15:02:09.559547+0000","last_became_active":"2024-04-25T14:59:45.329225+0000","last_became_peered":"2024-04-25T14:59:45.329225+0000","last_unstale":"2024-04-25T15:02:09.559547+0000","last_undegraded":"2024-04-25T15:02:09.559547+0000","last_fullsized":"2024-04-25T15:02:09.559547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:01.623458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574265+0000","last_change":"2024-04-25T14:59:46.320003+0000","last_active":"2024-04-25T15:02:09.574265+0000","last_peered":"2024-04-25T15:02:09.574265+0000","last_clean":"2024-04-25T15:02:09.574265+0000","last_became_active":"2024-04-25T14:59:46.319600+0000","last_became_peered":"2024-04-25T14:59:46.319600+0000","last_unstale":"2024-04-25T15:02:09.574265+0000","last_undegraded":"2024-04-25T15:02:09.574265+0000","last_fullsized":"2024-04-25T15:02:09.574265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:57.916018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426669+0000","last_change":"2024-04-25T14:59:45.325233+0000","last_active":"2024-04-25T15:02:09.426669+0000","last_peered":"2024-04-25T15:02:09.426669+0000","last_clean":"2024-04-25T15:02:09.426669+0000","last_became_active":"2024-04-25T14:59:45.324826+0000","last_became_peered":"2024-04-25T14:59:45.324826+0000","last_unstale":"2024-04-25T15:02:09.426669+0000","last_undegraded":"2024-04-25T15:02:09.426669+0000","last_fullsized":"2024-04-25T15:02:09.426669+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.422559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.574196+0000","last_change":"2024-04-25T14:59:46.314552+0000","last_active":"2024-04-25T15:02:09.574196+0000","last_peered":"2024-04-25T15:02:09.574196+0000","last_clean":"2024-04-25T15:02:09.574196+0000","last_became_active":"2024-04-25T14:59:46.314159+0000","last_became_peered":"2024-04-25T14:59:46.314159+0000","last_unstale":"2024-04-25T15:02:09.574196+0000","last_undegraded":"2024-04-25T15:02:09.574196+0000","last_fullsized":"2024-04-25T15:02:09.574196+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:06.731960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570613+0000","last_change":"2024-04-25T14:59:46.303354+0000","last_active":"2024-04-25T15:02:09.570613+0000","last_peered":"2024-04-25T15:02:09.570613+0000","last_clean":"2024-04-25T15:02:09.570613+0000","last_became_active":"2024-04-25T14:59:46.302869+0000","last_became_peered":"2024-04-25T14:59:46.302869+0000","last_unstale":"2024-04-25T15:02:09.570613+0000","last_undegraded":"2024-04-25T15:02:09.570613+0000","last_fullsized":"2024-04-25T15:02:09.570613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:03.796902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143216+0000","last_change":"2024-04-25T14:59:45.336686+0000","last_active":"2024-04-25T15:02:10.143216+0000","last_peered":"2024-04-25T15:02:10.143216+0000","last_clean":"2024-04-25T15:02:10.143216+0000","last_became_active":"2024-04-25T14:59:45.335828+0000","last_became_peered":"2024-04-25T14:59:45.335828+0000","last_unstale":"2024-04-25T15:02:10.143216+0000","last_undegraded":"2024-04-25T15:02:10.143216+0000","last_fullsized":"2024-04-25T15:02:10.143216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.676117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563154+0000","last_change":"2024-04-25T14:59:46.319541+0000","last_active":"2024-04-25T15:02:09.563154+0000","last_peered":"2024-04-25T15:02:09.563154+0000","last_clean":"2024-04-25T15:02:09.563154+0000","last_became_active":"2024-04-25T14:59:46.317922+0000","last_became_peered":"2024-04-25T14:59:46.317922+0000","last_unstale":"2024-04-25T15:02:09.563154+0000","last_undegraded":"2024-04-25T15:02:09.563154+0000","last_fullsized":"2024-04-25T15:02:09.563154+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:13.454075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457383+0000","last_change":"2024-04-25T14:59:45.313817+0000","last_active":"2024-04-25T15:02:09.457383+0000","last_peered":"2024-04-25T15:02:09.457383+0000","last_clean":"2024-04-25T15:02:09.457383+0000","last_became_active":"2024-04-25T14:59:45.313507+0000","last_became_peered":"2024-04-25T14:59:45.313507+0000","last_unstale":"2024-04-25T15:02:09.457383+0000","last_undegraded":"2024-04-25T15:02:09.457383+0000","last_fullsized":"2024-04-25T15:02:09.457383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:18.126351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":399,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:11.584390+0000","last_change":"2024-04-25T14:58:29.935018+0000","last_active":"2024-04-25T15:02:11.584390+0000","last_peered":"2024-04-25T15:02:11.584390+0000","last_clean":"2024-04-25T15:02:11.584390+0000","last_became_active":"2024-04-25T14:58:29.934454+0000","last_became_peered":"2024-04-25T14:58:29.934454+0000","last_unstale":"2024-04-25T15:02:11.584390+0000","last_undegraded":"2024-04-25T15:02:11.584390+0000","last_fullsized":"2024-04-25T15:02:11.584390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:54:46.455802+0000","last_clean_scrub_stamp":"2024-04-25T14:54:46.455802+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:09.308783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143505+0000","last_change":"2024-04-25T14:59:46.319680+0000","last_active":"2024-04-25T15:02:10.143505+0000","last_peered":"2024-04-25T15:02:10.143505+0000","last_clean":"2024-04-25T15:02:10.143505+0000","last_became_active":"2024-04-25T14:59:46.319225+0000","last_became_peered":"2024-04-25T14:59:46.319225+0000","last_unstale":"2024-04-25T15:02:10.143505+0000","last_undegraded":"2024-04-25T15:02:10.143505+0000","last_fullsized":"2024-04-25T15:02:10.143505+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:34.629244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426789+0000","last_change":"2024-04-25T14:59:45.318087+0000","last_active":"2024-04-25T15:02:09.426789+0000","last_peered":"2024-04-25T15:02:09.426789+0000","last_clean":"2024-04-25T15:02:09.426789+0000","last_became_active":"2024-04-25T14:59:45.317745+0000","last_became_peered":"2024-04-25T14:59:45.317745+0000","last_unstale":"2024-04-25T15:02:09.426789+0000","last_undegraded":"2024-04-25T15:02:09.426789+0000","last_fullsized":"2024-04-25T15:02:09.426789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:17.202201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562427+0000","last_change":"2024-04-25T14:59:45.332358+0000","last_active":"2024-04-25T15:02:09.562427+0000","last_peered":"2024-04-25T15:02:09.562427+0000","last_clean":"2024-04-25T15:02:09.562427+0000","last_became_active":"2024-04-25T14:59:45.332089+0000","last_became_peered":"2024-04-25T14:59:45.332089+0000","last_unstale":"2024-04-25T15:02:09.562427+0000","last_undegraded":"2024-04-25T15:02:09.562427+0000","last_fullsized":"2024-04-25T15:02:09.562427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:50.937548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143337+0000","last_change":"2024-04-25T14:59:46.320945+0000","last_active":"2024-04-25T15:02:10.143337+0000","last_peered":"2024-04-25T15:02:10.143337+0000","last_clean":"2024-04-25T15:02:10.143337+0000","last_became_active":"2024-04-25T14:59:46.320602+0000","last_became_peered":"2024-04-25T14:59:46.320602+0000","last_unstale":"2024-04-25T15:02:10.143337+0000","last_undegraded":"2024-04-25T15:02:10.143337+0000","last_fullsized":"2024-04-25T15:02:10.143337+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:32.282647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'96","reported_seq":210,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:12.018657+0000","last_change":"2024-04-25T14:59:45.334325+0000","last_active":"2024-04-25T15:02:12.018657+0000","last_peered":"2024-04-25T15:02:12.018657+0000","last_clean":"2024-04-25T15:02:12.018657+0000","last_became_active":"2024-04-25T14:59:45.333969+0000","last_became_peered":"2024-04-25T14:59:45.333969+0000","last_unstale":"2024-04-25T15:02:12.018657+0000","last_undegraded":"2024-04-25T15:02:12.018657+0000","last_fullsized":"2024-04-25T15:02:12.018657+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:02.694267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":176,"num_write_kb":1285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.559772+0000","last_change":"2024-04-25T15:00:02.726558+0000","last_active":"2024-04-25T15:02:09.559772+0000","last_peered":"2024-04-25T15:02:09.559772+0000","last_clean":"2024-04-25T15:02:09.559772+0000","last_became_active":"2024-04-25T15:00:02.725467+0000","last_became_peered":"2024-04-25T15:00:02.725467+0000","last_unstale":"2024-04-25T15:02:09.559772+0000","last_undegraded":"2024-04-25T15:02:09.559772+0000","last_fullsized":"2024-04-25T15:02:09.559772+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:44.508211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.457534+0000","last_change":"2024-04-25T14:59:45.325553+0000","last_active":"2024-04-25T15:02:09.457534+0000","last_peered":"2024-04-25T15:02:09.457534+0000","last_clean":"2024-04-25T15:02:09.457534+0000","last_became_active":"2024-04-25T14:59:45.325201+0000","last_became_peered":"2024-04-25T14:59:45.325201+0000","last_unstale":"2024-04-25T15:02:09.457534+0000","last_undegraded":"2024-04-25T15:02:09.457534+0000","last_fullsized":"2024-04-25T15:02:09.457534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:05.068269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427983+0000","last_change":"2024-04-25T14:59:46.318252+0000","last_active":"2024-04-25T15:02:09.427983+0000","last_peered":"2024-04-25T15:02:09.427983+0000","last_clean":"2024-04-25T15:02:09.427983+0000","last_became_active":"2024-04-25T14:59:46.317601+0000","last_became_peered":"2024-04-25T14:59:46.317601+0000","last_unstale":"2024-04-25T15:02:09.427983+0000","last_undegraded":"2024-04-25T15:02:09.427983+0000","last_fullsized":"2024-04-25T15:02:09.427983+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:33.847928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.739599+0000","last_change":"2024-04-25T14:59:45.329430+0000","last_active":"2024-04-25T15:02:09.739599+0000","last_peered":"2024-04-25T15:02:09.739599+0000","last_clean":"2024-04-25T15:02:09.739599+0000","last_became_active":"2024-04-25T14:59:45.329140+0000","last_became_peered":"2024-04-25T14:59:45.329140+0000","last_unstale":"2024-04-25T15:02:09.739599+0000","last_undegraded":"2024-04-25T15:02:09.739599+0000","last_fullsized":"2024-04-25T15:02:09.739599+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:49.114820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562082+0000","last_change":"2024-04-25T14:59:46.299821+0000","last_active":"2024-04-25T15:02:09.562082+0000","last_peered":"2024-04-25T15:02:09.562082+0000","last_clean":"2024-04-25T15:02:09.562082+0000","last_became_active":"2024-04-25T14:59:46.299511+0000","last_became_peered":"2024-04-25T14:59:46.299511+0000","last_unstale":"2024-04-25T15:02:09.562082+0000","last_undegraded":"2024-04-25T15:02:09.562082+0000","last_fullsized":"2024-04-25T15:02:09.562082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:23.409333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562929+0000","last_change":"2024-04-25T14:59:45.331464+0000","last_active":"2024-04-25T15:02:09.562929+0000","last_peered":"2024-04-25T15:02:09.562929+0000","last_clean":"2024-04-25T15:02:09.562929+0000","last_became_active":"2024-04-25T14:59:45.327879+0000","last_became_peered":"2024-04-25T14:59:45.327879+0000","last_unstale":"2024-04-25T15:02:09.562929+0000","last_undegraded":"2024-04-25T15:02:09.562929+0000","last_fullsized":"2024-04-25T15:02:09.562929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:06:50.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818825+0000","last_change":"2024-04-25T14:59:46.310621+0000","last_active":"2024-04-25T15:02:09.818825+0000","last_peered":"2024-04-25T15:02:09.818825+0000","last_clean":"2024-04-25T15:02:09.818825+0000","last_became_active":"2024-04-25T14:59:46.310292+0000","last_became_peered":"2024-04-25T14:59:46.310292+0000","last_unstale":"2024-04-25T15:02:09.818825+0000","last_undegraded":"2024-04-25T15:02:09.818825+0000","last_fullsized":"2024-04-25T15:02:09.818825+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:20.035121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818912+0000","last_change":"2024-04-25T15:00:02.713741+0000","last_active":"2024-04-25T15:02:09.818912+0000","last_peered":"2024-04-25T15:02:09.818912+0000","last_clean":"2024-04-25T15:02:09.818912+0000","last_became_active":"2024-04-25T15:00:02.713336+0000","last_became_peered":"2024-04-25T15:00:02.713336+0000","last_unstale":"2024-04-25T15:02:09.818912+0000","last_undegraded":"2024-04-25T15:02:09.818912+0000","last_fullsized":"2024-04-25T15:02:09.818912+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:37.967019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563077+0000","last_change":"2024-04-25T14:59:46.326568+0000","last_active":"2024-04-25T15:02:09.563077+0000","last_peered":"2024-04-25T15:02:09.563077+0000","last_clean":"2024-04-25T15:02:09.563077+0000","last_became_active":"2024-04-25T14:59:46.324019+0000","last_became_peered":"2024-04-25T14:59:46.324019+0000","last_unstale":"2024-04-25T15:02:09.563077+0000","last_undegraded":"2024-04-25T15:02:09.563077+0000","last_fullsized":"2024-04-25T15:02:09.563077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:27.556126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.562234+0000","last_change":"2024-04-25T14:59:45.316935+0000","last_active":"2024-04-25T15:02:09.562234+0000","last_peered":"2024-04-25T15:02:09.562234+0000","last_clean":"2024-04-25T15:02:09.562234+0000","last_became_active":"2024-04-25T14:59:45.316604+0000","last_became_peered":"2024-04-25T14:59:45.316604+0000","last_unstale":"2024-04-25T15:02:09.562234+0000","last_undegraded":"2024-04-25T15:02:09.562234+0000","last_fullsized":"2024-04-25T15:02:09.562234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:22:45.699926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426847+0000","last_change":"2024-04-25T14:59:46.316195+0000","last_active":"2024-04-25T15:02:09.426847+0000","last_peered":"2024-04-25T15:02:09.426847+0000","last_clean":"2024-04-25T15:02:09.426847+0000","last_became_active":"2024-04-25T14:59:46.315809+0000","last_became_peered":"2024-04-25T14:59:46.315809+0000","last_unstale":"2024-04-25T15:02:09.426847+0000","last_undegraded":"2024-04-25T15:02:09.426847+0000","last_fullsized":"2024-04-25T15:02:09.426847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:34.440058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426952+0000","last_change":"2024-04-25T14:59:45.325137+0000","last_active":"2024-04-25T15:02:09.426952+0000","last_peered":"2024-04-25T15:02:09.426952+0000","last_clean":"2024-04-25T15:02:09.426952+0000","last_became_active":"2024-04-25T14:59:45.324769+0000","last_became_peered":"2024-04-25T14:59:45.324769+0000","last_unstale":"2024-04-25T15:02:09.426952+0000","last_undegraded":"2024-04-25T15:02:09.426952+0000","last_fullsized":"2024-04-25T15:02:09.426952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:02.778853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427258+0000","last_change":"2024-04-25T14:59:46.304099+0000","last_active":"2024-04-25T15:02:09.427258+0000","last_peered":"2024-04-25T15:02:09.427258+0000","last_clean":"2024-04-25T15:02:09.427258+0000","last_became_active":"2024-04-25T14:59:46.303774+0000","last_became_peered":"2024-04-25T14:59:46.303774+0000","last_unstale":"2024-04-25T15:02:09.427258+0000","last_undegraded":"2024-04-25T15:02:09.427258+0000","last_fullsized":"2024-04-25T15:02:09.427258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:45.308212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'65","reported_seq":118,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:11.551625+0000","last_change":"2024-04-25T14:59:45.322324+0000","last_active":"2024-04-25T15:02:11.551625+0000","last_peered":"2024-04-25T15:02:11.551625+0000","last_clean":"2024-04-25T15:02:11.551625+0000","last_became_active":"2024-04-25T14:59:45.322031+0000","last_became_peered":"2024-04-25T14:59:45.322031+0000","last_unstale":"2024-04-25T15:02:11.551625+0000","last_undegraded":"2024-04-25T15:02:11.551625+0000","last_fullsized":"2024-04-25T15:02:11.551625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:52.424928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'7","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427253+0000","last_change":"2024-04-25T15:00:02.730051+0000","last_active":"2024-04-25T15:02:09.427253+0000","last_peered":"2024-04-25T15:02:09.427253+0000","last_clean":"2024-04-25T15:02:09.427253+0000","last_became_active":"2024-04-25T15:00:02.725610+0000","last_became_peered":"2024-04-25T15:00:02.725610+0000","last_unstale":"2024-04-25T15:02:09.427253+0000","last_undegraded":"2024-04-25T15:02:09.427253+0000","last_fullsized":"2024-04-25T15:02:09.427253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:04.598366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143595+0000","last_change":"2024-04-25T14:59:45.335502+0000","last_active":"2024-04-25T15:02:10.143595+0000","last_peered":"2024-04-25T15:02:10.143595+0000","last_clean":"2024-04-25T15:02:10.143595+0000","last_became_active":"2024-04-25T14:59:45.335018+0000","last_became_peered":"2024-04-25T14:59:45.335018+0000","last_unstale":"2024-04-25T15:02:10.143595+0000","last_undegraded":"2024-04-25T15:02:10.143595+0000","last_fullsized":"2024-04-25T15:02:10.143595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:44.286988+0000","last_clean_scrub_stamp":"2024-04-25T14:59:44.286988+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:48:11.918001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.427140+0000","last_change":"2024-04-25T14:59:46.304766+0000","last_active":"2024-04-25T15:02:09.427140+0000","last_peered":"2024-04-25T15:02:09.427140+0000","last_clean":"2024-04-25T15:02:09.427140+0000","last_became_active":"2024-04-25T14:59:46.304413+0000","last_became_peered":"2024-04-25T14:59:46.304413+0000","last_unstale":"2024-04-25T15:02:09.427140+0000","last_undegraded":"2024-04-25T15:02:09.427140+0000","last_fullsized":"2024-04-25T15:02:09.427140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:50.766228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818984+0000","last_change":"2024-04-25T14:59:46.305285+0000","last_active":"2024-04-25T15:02:09.818984+0000","last_peered":"2024-04-25T15:02:09.818984+0000","last_clean":"2024-04-25T15:02:09.818984+0000","last_became_active":"2024-04-25T14:59:46.304934+0000","last_became_peered":"2024-04-25T14:59:46.304934+0000","last_unstale":"2024-04-25T15:02:09.818984+0000","last_undegraded":"2024-04-25T15:02:09.818984+0000","last_fullsized":"2024-04-25T15:02:09.818984+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:44.892921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426447+0000","last_change":"2024-04-25T15:00:02.718240+0000","last_active":"2024-04-25T15:02:09.426447+0000","last_peered":"2024-04-25T15:02:09.426447+0000","last_clean":"2024-04-25T15:02:09.426447+0000","last_became_active":"2024-04-25T15:00:02.717792+0000","last_became_peered":"2024-04-25T15:00:02.717792+0000","last_unstale":"2024-04-25T15:02:09.426447+0000","last_undegraded":"2024-04-25T15:02:09.426447+0000","last_fullsized":"2024-04-25T15:02:09.426447+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:49.532532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426395+0000","last_change":"2024-04-25T14:59:46.319694+0000","last_active":"2024-04-25T15:02:09.426395+0000","last_peered":"2024-04-25T15:02:09.426395+0000","last_clean":"2024-04-25T15:02:09.426395+0000","last_became_active":"2024-04-25T14:59:46.319417+0000","last_became_peered":"2024-04-25T14:59:46.319417+0000","last_unstale":"2024-04-25T15:02:09.426395+0000","last_undegraded":"2024-04-25T15:02:09.426395+0000","last_fullsized":"2024-04-25T15:02:09.426395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:48.177498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.428062+0000","last_change":"2024-04-25T15:00:02.986849+0000","last_active":"2024-04-25T15:02:09.428062+0000","last_peered":"2024-04-25T15:02:09.428062+0000","last_clean":"2024-04-25T15:02:09.428062+0000","last_became_active":"2024-04-25T15:00:02.986345+0000","last_became_peered":"2024-04-25T15:00:02.986345+0000","last_unstale":"2024-04-25T15:02:09.428062+0000","last_undegraded":"2024-04-25T15:02:09.428062+0000","last_fullsized":"2024-04-25T15:02:09.428062+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:22.707469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143409+0000","last_change":"2024-04-25T14:59:46.319167+0000","last_active":"2024-04-25T15:02:10.143409+0000","last_peered":"2024-04-25T15:02:10.143409+0000","last_clean":"2024-04-25T15:02:10.143409+0000","last_became_active":"2024-04-25T14:59:46.318804+0000","last_became_peered":"2024-04-25T14:59:46.318804+0000","last_unstale":"2024-04-25T15:02:10.143409+0000","last_undegraded":"2024-04-25T15:02:10.143409+0000","last_fullsized":"2024-04-25T15:02:10.143409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:34.503979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818946+0000","last_change":"2024-04-25T14:59:46.304554+0000","last_active":"2024-04-25T15:02:09.818946+0000","last_peered":"2024-04-25T15:02:09.818946+0000","last_clean":"2024-04-25T15:02:09.818946+0000","last_became_active":"2024-04-25T14:59:46.304062+0000","last_became_peered":"2024-04-25T14:59:46.304062+0000","last_unstale":"2024-04-25T15:02:09.818946+0000","last_undegraded":"2024-04-25T15:02:09.818946+0000","last_fullsized":"2024-04-25T15:02:09.818946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:03.670917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.426897+0000","last_change":"2024-04-25T14:59:46.299104+0000","last_active":"2024-04-25T15:02:09.426897+0000","last_peered":"2024-04-25T15:02:09.426897+0000","last_clean":"2024-04-25T15:02:09.426897+0000","last_became_active":"2024-04-25T14:59:46.298765+0000","last_became_peered":"2024-04-25T14:59:46.298765+0000","last_unstale":"2024-04-25T15:02:09.426897+0000","last_undegraded":"2024-04-25T15:02:09.426897+0000","last_fullsized":"2024-04-25T15:02:09.426897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:00.494298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563297+0000","last_change":"2024-04-25T14:59:46.328885+0000","last_active":"2024-04-25T15:02:09.563297+0000","last_peered":"2024-04-25T15:02:09.563297+0000","last_clean":"2024-04-25T15:02:09.563297+0000","last_became_active":"2024-04-25T14:59:46.328375+0000","last_became_peered":"2024-04-25T14:59:46.328375+0000","last_unstale":"2024-04-25T15:02:09.563297+0000","last_undegraded":"2024-04-25T15:02:09.563297+0000","last_fullsized":"2024-04-25T15:02:09.563297+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:08.959045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.570545+0000","last_change":"2024-04-25T14:59:46.317403+0000","last_active":"2024-04-25T15:02:09.570545+0000","last_peered":"2024-04-25T15:02:09.570545+0000","last_clean":"2024-04-25T15:02:09.570545+0000","last_became_active":"2024-04-25T14:59:46.317100+0000","last_became_peered":"2024-04-25T14:59:46.317100+0000","last_unstale":"2024-04-25T15:02:09.570545+0000","last_undegraded":"2024-04-25T15:02:09.570545+0000","last_fullsized":"2024-04-25T15:02:09.570545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:24:23.694595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.563030+0000","last_change":"2024-04-25T14:59:46.327147+0000","last_active":"2024-04-25T15:02:09.563030+0000","last_peered":"2024-04-25T15:02:09.563030+0000","last_clean":"2024-04-25T15:02:09.563030+0000","last_became_active":"2024-04-25T14:59:46.321857+0000","last_became_peered":"2024-04-25T14:59:46.321857+0000","last_unstale":"2024-04-25T15:02:09.563030+0000","last_undegraded":"2024-04-25T15:02:09.563030+0000","last_fullsized":"2024-04-25T15:02:09.563030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:46.280388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.740053+0000","last_change":"2024-04-25T14:59:46.310841+0000","last_active":"2024-04-25T15:02:09.740053+0000","last_peered":"2024-04-25T15:02:09.740053+0000","last_clean":"2024-04-25T15:02:09.740053+0000","last_became_active":"2024-04-25T14:59:46.310446+0000","last_became_peered":"2024-04-25T14:59:46.310446+0000","last_unstale":"2024-04-25T15:02:09.740053+0000","last_undegraded":"2024-04-25T15:02:09.740053+0000","last_fullsized":"2024-04-25T15:02:09.740053+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:11.594784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:10.143711+0000","last_change":"2024-04-25T14:59:46.320715+0000","last_active":"2024-04-25T15:02:10.143711+0000","last_peered":"2024-04-25T15:02:10.143711+0000","last_clean":"2024-04-25T15:02:10.143711+0000","last_became_active":"2024-04-25T14:59:46.320238+0000","last_became_peered":"2024-04-25T14:59:46.320238+0000","last_unstale":"2024-04-25T15:02:10.143711+0000","last_undegraded":"2024-04-25T15:02:10.143711+0000","last_fullsized":"2024-04-25T15:02:10.143711+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:55.387814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T15:02:09.818885+0000","last_change":"2024-04-25T14:59:46.303772+0000","last_active":"2024-04-25T15:02:09.818885+0000","last_peered":"2024-04-25T15:02:09.818885+0000","last_clean":"2024-04-25T15:02:09.818885+0000","last_became_active":"2024-04-25T14:59:46.303435+0000","last_became_peered":"2024-04-25T14:59:46.303435+0000","last_unstale":"2024-04-25T15:02:09.818885+0000","last_undegraded":"2024-04-25T15:02:09.818885+0000","last_fullsized":"2024-04-25T15:02:09.818885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T14:59:45.293770+0000","last_clean_scrub_stamp":"2024-04-25T14:59:45.293770+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:06.692979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":790006,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":663,"num_read_kb":537,"num_write":322,"num_write_kb":2317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2469888,"data_stored":2370018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":970752,"data_stored":467709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29080,"kb_used_data":1944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716072,"statfs":{"total":95995035648,"available":95965257728,"internally_reserved":0,"allocated":1990656,"data_stored":1454380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36680,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708472,"statfs":{"total":95995035648,"available":95957475328,"internally_reserved":0,"allocated":1372160,"data_stored":861624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037819,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28356,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716796,"statfs":{"total":95995035648,"available":95965999104,"internally_reserved":0,"allocated":1372160,"data_stored":861529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299456,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1572864,"data_stored":1058776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462983,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":467826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1048576,"data_stored":535469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":467799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411422,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":535503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149783,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28804,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716348,"statfs":{"total":95995035648,"available":95965540352,"internally_reserved":0,"allocated":1708032,"data_stored":1189378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888146,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":974848,"data_stored":467731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724619,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29004,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716148,"statfs":{"total":95995035648,"available":95965335552,"internally_reserved":0,"allocated":1912832,"data_stored":1386608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":733184,"data_stored":721552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":328414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":393820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":396186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:02:23.211 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:02:23.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:23 smithi094 ceph-mon[27862]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.460+0000 7f171a800640 1 -- 172.21.15.94:0/226171373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 msgr2=0x7f171410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 --2- 172.21.15.94:0/226171373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f171410ee30 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f170c009ca0 tx=0x7f170c02a8c0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 -- 172.21.15.94:0/226171373 shutdown_connections 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 --2- 172.21.15.94:0/226171373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f171410ee30 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 --2- 172.21.15.94:0/226171373 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1714102b40 0x7f1714106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 --2- 172.21.15.94:0/226171373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1714102190 0x7f1714102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 -- 172.21.15.94:0/226171373 >> 172.21.15.94:0/226171373 conn(0x7f1714079310 msgr2=0x7f1714079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 -- 172.21.15.94:0/226171373 shutdown_connections 2024-04-25T15:02:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.461+0000 7f171a800640 1 -- 172.21.15.94:0/226171373 wait complete. 2024-04-25T15:02:23.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.462+0000 7f171a800640 1 Processor -- start 2024-04-25T15:02:23.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.462+0000 7f171a800640 1 -- start start 2024-04-25T15:02:23.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f171a800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1714102190 0x7f1714198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f171a800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f1714198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f171a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f171419cff0 0x7f171419d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f171a800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f17141121d0 con 0x7f1714107260 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f171a800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f1714112050 con 0x7f1714102190 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f171a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1714112350 con 0x7f171419cff0 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f1714198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f1714198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52352/0 (socket says 172.21.15.94:52352) 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 -- 172.21.15.94:0/2982707534 learned_addr learned my addr 172.21.15.94:0/2982707534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1713e00640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f171419cff0 0x7f171419d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1713400640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1714102190 0x7f1714198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 -- 172.21.15.94:0/2982707534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f171419cff0 msgr2=0x7f171419d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f171419cff0 0x7f171419d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 -- 172.21.15.94:0/2982707534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1714102190 msgr2=0x7f1714198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1714102190 0x7f1714198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.463+0000 7f1712a00640 1 -- 172.21.15.94:0/2982707534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171419dae0 con 0x7f1714107260 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.464+0000 7f1712a00640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f1714198c70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e9a0 tx=0x7f16fc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.464+0000 7f170be00640 1 -- 172.21.15.94:0/2982707534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc00cd80 con 0x7f1714107260 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.464+0000 7f170be00640 1 -- 172.21.15.94:0/2982707534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16fc0035f0 con 0x7f1714107260 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.464+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171419ddd0 con 0x7f1714107260 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.464+0000 7f170be00640 1 -- 172.21.15.94:0/2982707534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc010690 con 0x7f1714107260 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.464+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17141a55c0 con 0x7f1714107260 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.465+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17140691c0 con 0x7f1714107260 2024-04-25T15:02:23.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.467+0000 7f170be00640 1 -- 172.21.15.94:0/2982707534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f16fc004000 con 0x7f1714107260 2024-04-25T15:02:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.469+0000 7f170be00640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16f80a3db0 0x7f16f80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.469+0000 7f170be00640 1 -- 172.21.15.94:0/2982707534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f16fc0d0c10 con 0x7f1714107260 2024-04-25T15:02:23.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.469+0000 7f1713400640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16f80a3db0 0x7f16f80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.470+0000 7f1713400640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16f80a3db0 0x7f16f80a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1704009bd0 tx=0x7f1704005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:23.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.473+0000 7f170be00640 1 -- 172.21.15.94:0/2982707534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f16fc0042f0 con 0x7f1714107260 2024-04-25T15:02:23.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:23 smithi137 ceph-mon[32439]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T15:02:23.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.676+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1714102b40 con 0x7f1714107260 2024-04-25T15:02:23.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.678+0000 7f170be00640 1 -- 172.21.15.94:0/2982707534 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23061 (secure 0 0 0) 0x7f16fc09d0f0 con 0x7f1714107260 2024-04-25T15:02:23.680 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:23.680 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":89,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:02:09.415151+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/702744035":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3354212797":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6801/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1994260887":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/2523030980":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6800/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16f80a3db0 msgr2=0x7f16f80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16f80a3db0 0x7f16f80a6270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1704009bd0 tx=0x7f1704005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 msgr2=0x7f1714198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f1714198c70 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f16fc00e9a0 tx=0x7f16fc00ee60 comp rx=0 tx=0).stop 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 shutdown_connections 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16f80a3db0 0x7f16f80a6270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f171419cff0 0x7f171419d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1714107260 0x7f1714198c70 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 --2- 172.21.15.94:0/2982707534 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f1714102190 0x7f1714198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 >> 172.21.15.94:0/2982707534 conn(0x7f1714079310 msgr2=0x7f1714100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 shutdown_connections 2024-04-25T15:02:23.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.681+0000 7f171a800640 1 -- 172.21.15.94:0/2982707534 wait complete. 2024-04-25T15:02:23.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:02:23.702 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:02:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:23 smithi138 ceph-mon[32875]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 83 KiB/s wr, 47 op/s 2024-04-25T15:02:23.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 -- 172.21.15.94:0/3604281857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90112c90 msgr2=0x7f0e90115080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3604281857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90112c90 0x7f0e90115080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e88009c10 tx=0x7f0e8802a6f0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 -- 172.21.15.94:0/3604281857 shutdown_connections 2024-04-25T15:02:23.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3604281857 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90112c90 0x7f0e90115080 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3604281857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076da0 0x7f0e90112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3604281857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e90076480 0x7f0e90076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 -- 172.21.15.94:0/3604281857 >> 172.21.15.94:0/3604281857 conn(0x7f0e900ffac0 msgr2=0x7f0e90101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:23.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 -- 172.21.15.94:0/3604281857 shutdown_connections 2024-04-25T15:02:23.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.900+0000 7f0e8f200640 1 -- 172.21.15.94:0/3604281857 wait complete. 2024-04-25T15:02:23.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 Processor -- start 2024-04-25T15:02:23.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 -- start start 2024-04-25T15:02:23.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076480 0x7f0e901a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90076da0 0x7f0e901a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e90112c90 0x7f0e9019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f0e90118660 con 0x7f0e90112c90 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901184e0 con 0x7f0e90076da0 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.902+0000 7f0e8f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e901187e0 con 0x7f0e90076480 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076480 0x7f0e901a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8e800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e90112c90 0x7f0e9019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076480 0x7f0e901a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:49984/0 (socket says 172.21.15.94:49984) 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 -- 172.21.15.94:0/3282512802 learned_addr learned my addr 172.21.15.94:0/3282512802 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8d400640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90076da0 0x7f0e901a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 -- 172.21.15.94:0/3282512802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90076da0 msgr2=0x7f0e901a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90076da0 0x7f0e901a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 -- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e90112c90 msgr2=0x7f0e9019eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:23.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e90112c90 0x7f0e9019eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:23.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 -- 172.21.15.94:0/3282512802 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9019f230 con 0x7f0e90076480 2024-04-25T15:02:23.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8e800640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e90112c90 0x7f0e9019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:23.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.903+0000 7f0e8de00640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076480 0x7f0e901a48b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000e9c0 tx=0x7f0e8000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:23.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.904+0000 7f0e86a00640 1 -- 172.21.15.94:0/3282512802 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000ccc0 con 0x7f0e90076480 2024-04-25T15:02:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.904+0000 7f0e86a00640 1 -- 172.21.15.94:0/3282512802 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e800054d0 con 0x7f0e90076480 2024-04-25T15:02:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.904+0000 7f0e86a00640 1 -- 172.21.15.94:0/3282512802 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e80010430 con 0x7f0e90076480 2024-04-25T15:02:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.904+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9019f520 con 0x7f0e90076480 2024-04-25T15:02:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.904+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e901ab770 con 0x7f0e90076480 2024-04-25T15:02:23.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.905+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e900690e0 con 0x7f0e90076480 2024-04-25T15:02:23.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.907+0000 7f0e86a00640 1 -- 172.21.15.94:0/3282512802 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f0e80004f80 con 0x7f0e90076480 2024-04-25T15:02:23.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.908+0000 7f0e86a00640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f0e740a3db0 0x7f0e740a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.908+0000 7f0e86a00640 1 -- 172.21.15.94:0/3282512802 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f0e800d0db0 con 0x7f0e90076480 2024-04-25T15:02:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.909+0000 7f0e8d400640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f0e740a3db0 0x7f0e740a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:23.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.909+0000 7f0e8d400640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f0e740a3db0 0x7f0e740a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0095f0 tx=0x7f0e7c005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:23.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:23.911+0000 7f0e86a00640 1 -- 172.21.15.94:0/3282512802 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e8009d290 con 0x7f0e90076480 2024-04-25T15:02:24.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.116+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0e9019ff90 con 0x7f0e90076480 2024-04-25T15:02:24.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.117+0000 7f0e86a00640 1 -- 172.21.15.94:0/3282512802 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f0e8009cb90 con 0x7f0e90076480 2024-04-25T15:02:24.118 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:02:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f0e740a3db0 msgr2=0x7f0e740a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f0e740a3db0 0x7f0e740a6270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0095f0 tx=0x7f0e7c005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:24.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076480 msgr2=0x7f0e901a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076480 0x7f0e901a48b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0e8000e9c0 tx=0x7f0e8000ee80 comp rx=0 tx=0).stop 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 shutdown_connections 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f0e740a3db0 0x7f0e740a6270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0e90112c90 0x7f0e9019eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0e90076da0 0x7f0e901a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 --2- 172.21.15.94:0/3282512802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e90076480 0x7f0e901a48b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 >> 172.21.15.94:0/3282512802 conn(0x7f0e900ffac0 msgr2=0x7f0e90101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 shutdown_connections 2024-04-25T15:02:24.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.120+0000 7f0e8f200640 1 -- 172.21.15.94:0/3282512802 wait complete. 2024-04-25T15:02:24.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:02:24.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 -- 172.21.15.94:0/2038369984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a4108e90 msgr2=0x7f16a41152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 --2- 172.21.15.94:0/2038369984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a4108e90 0x7f16a41152b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f169c009ca0 tx=0x7f169c02a950 comp rx=0 tx=0).stop 2024-04-25T15:02:24.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 -- 172.21.15.94:0/2038369984 shutdown_connections 2024-04-25T15:02:24.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 --2- 172.21.15.94:0/2038369984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a4108e90 0x7f16a41152b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 --2- 172.21.15.94:0/2038369984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a41084f0 0x7f16a4108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 --2- 172.21.15.94:0/2038369984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a410d3c0 0x7f16a410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 -- 172.21.15.94:0/2038369984 >> 172.21.15.94:0/2038369984 conn(0x7f16a4077d70 msgr2=0x7f16a4106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:24.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 -- 172.21.15.94:0/2038369984 shutdown_connections 2024-04-25T15:02:24.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.350+0000 7f16a3200640 1 -- 172.21.15.94:0/2038369984 wait complete. 2024-04-25T15:02:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.351+0000 7f16a3200640 1 Processor -- start 2024-04-25T15:02:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.351+0000 7f16a3200640 1 -- start start 2024-04-25T15:02:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a41084f0 0x7f16a41a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a4108e90 0x7f16a41a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a410d3c0 0x7f16a419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f16a41186b0 con 0x7f16a41084f0 2024-04-25T15:02:24.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4118530 con 0x7f16a410d3c0 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16a4118830 con 0x7f16a4108e90 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a41084f0 0x7f16a41a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a41084f0 0x7f16a41a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52390/0 (socket says 172.21.15.94:52390) 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a1e00640 1 -- 172.21.15.94:0/2676169984 learned_addr learned my addr 172.21.15.94:0/2676169984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f1699400640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a4108e90 0x7f16a41a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.352+0000 7f16a2800640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a410d3c0 0x7f16a419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a1e00640 1 -- 172.21.15.94:0/2676169984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a4108e90 msgr2=0x7f16a41a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a1e00640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a4108e90 0x7f16a41a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a1e00640 1 -- 172.21.15.94:0/2676169984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a410d3c0 msgr2=0x7f16a419ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a1e00640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a410d3c0 0x7f16a419ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a1e00640 1 -- 172.21.15.94:0/2676169984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a419f1b0 con 0x7f16a41084f0 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f1699400640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a4108e90 0x7f16a41a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a1e00640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a41084f0 0x7f16a41a4880 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f169400e9f0 tx=0x7f169400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:24.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f169b400640 1 -- 172.21.15.94:0/2676169984 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169400cd80 con 0x7f16a41084f0 2024-04-25T15:02:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f169b400640 1 -- 172.21.15.94:0/2676169984 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16940108e0 con 0x7f16a41084f0 2024-04-25T15:02:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a419f4a0 con 0x7f16a41084f0 2024-04-25T15:02:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f169b400640 1 -- 172.21.15.94:0/2676169984 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1694018980 con 0x7f16a41084f0 2024-04-25T15:02:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.353+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a41ab740 con 0x7f16a41084f0 2024-04-25T15:02:24.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.355+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a40690e0 con 0x7f16a41084f0 2024-04-25T15:02:24.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.357+0000 7f169b400640 1 -- 172.21.15.94:0/2676169984 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1694010430 con 0x7f16a41084f0 2024-04-25T15:02:24.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.358+0000 7f169b400640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16880a3db0 0x7f16880a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.359+0000 7f169b400640 1 -- 172.21.15.94:0/2676169984 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f16940d0c50 con 0x7f16a41084f0 2024-04-25T15:02:24.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.359+0000 7f1699400640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16880a3db0 0x7f16880a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.359+0000 7f1699400640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16880a3db0 0x7f16880a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16a4108040 tx=0x7f168c005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:24.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.362+0000 7f169b400640 1 -- 172.21.15.94:0/2676169984 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f169409cae0 con 0x7f16a41084f0 2024-04-25T15:02:24.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:24 smithi094 ceph-mon[27862]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:24.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:24 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2982707534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:24.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:24 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3282512802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:02:24.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.564+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f16a4110900 con 0x7f16a41084f0 2024-04-25T15:02:24.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.565+0000 7f169b400640 1 -- 172.21.15.94:0/2676169984 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f16940a0de0 con 0x7f16a41084f0 2024-04-25T15:02:24.566 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:02:24.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.567+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16880a3db0 msgr2=0x7f16880a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.567+0000 7f16a3200640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16880a3db0 0x7f16880a6270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f16a4108040 tx=0x7f168c005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:24.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.567+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a41084f0 msgr2=0x7f16a41a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.567+0000 7f16a3200640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a41084f0 0x7f16a41a4880 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f169400e9f0 tx=0x7f169400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 shutdown_connections 2024-04-25T15:02:24.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f16880a3db0 0x7f16880a6270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f16a410d3c0 0x7f16a419ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a4108e90 0x7f16a41a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 --2- 172.21.15.94:0/2676169984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f16a41084f0 0x7f16a41a4880 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 >> 172.21.15.94:0/2676169984 conn(0x7f16a4077d70 msgr2=0x7f16a40783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:24.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 shutdown_connections 2024-04-25T15:02:24.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.568+0000 7f16a3200640 1 -- 172.21.15.94:0/2676169984 wait complete. 2024-04-25T15:02:24.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:02:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:24 smithi138 ceph-mon[32875]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:24 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2982707534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:24 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3282512802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:02:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:24 smithi137 ceph-mon[32439]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:02:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:24 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2982707534' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:24.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:24 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3282512802' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 -- 172.21.15.94:0/3242894045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102a10 msgr2=0x7fa2bc106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/3242894045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102a10 0x7fa2bc106a50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009ca0 tx=0x7fa2ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 -- 172.21.15.94:0/3242894045 shutdown_connections 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/3242894045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2bc107130 0x7fa2bc10ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/3242894045 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102a10 0x7fa2bc106a50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/3242894045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc102060 0x7fa2bc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 -- 172.21.15.94:0/3242894045 >> 172.21.15.94:0/3242894045 conn(0x7fa2bc0fd920 msgr2=0x7fa2bc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 -- 172.21.15.94:0/3242894045 shutdown_connections 2024-04-25T15:02:24.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.783+0000 7fa2c0e00640 1 -- 172.21.15.94:0/3242894045 wait complete. 2024-04-25T15:02:24.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 Processor -- start 2024-04-25T15:02:24.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 -- start start 2024-04-25T15:02:24.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102060 0x7fa2bc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2bc107130 0x7fa2bc198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc19cf60 0x7fa2bc19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc112070 con 0x7fa2bc107130 2024-04-25T15:02:24.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc111ef0 con 0x7fa2bc102060 2024-04-25T15:02:24.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.785+0000 7fa2c0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc1121f0 con 0x7fa2bc19cf60 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102060 0x7fa2bc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc19cf60 0x7fa2bc19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102060 0x7fa2bc1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43292/0 (socket says 172.21.15.94:43292) 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 -- 172.21.15.94:0/953387880 learned_addr learned my addr 172.21.15.94:0/953387880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2baa00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2bc107130 0x7fa2bc198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 -- 172.21.15.94:0/953387880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc19cf60 msgr2=0x7fa2bc19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc19cf60 0x7fa2bc19d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 -- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2bc107130 msgr2=0x7fa2bc198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2bc107130 0x7fa2bc198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:24.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 -- 172.21.15.94:0/953387880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc19da50 con 0x7fa2bc102060 2024-04-25T15:02:24.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.786+0000 7fa2bb400640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102060 0x7fa2bc1986a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9f0 tx=0x7fa2a400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:24.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.787+0000 7fa2b3e00640 1 -- 172.21.15.94:0/953387880 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400cd80 con 0x7fa2bc102060 2024-04-25T15:02:24.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.787+0000 7fa2b3e00640 1 -- 172.21.15.94:0/953387880 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a40054d0 con 0x7fa2bc102060 2024-04-25T15:02:24.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.787+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc19dd40 con 0x7fa2bc102060 2024-04-25T15:02:24.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.787+0000 7fa2b3e00640 1 -- 172.21.15.94:0/953387880 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4010690 con 0x7fa2bc102060 2024-04-25T15:02:24.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.787+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc1a5500 con 0x7fa2bc102060 2024-04-25T15:02:24.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.788+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc068c90 con 0x7fa2bc102060 2024-04-25T15:02:24.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.790+0000 7fa2b3e00640 1 -- 172.21.15.94:0/953387880 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa2a4004e30 con 0x7fa2bc102060 2024-04-25T15:02:24.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.791+0000 7fa2b3e00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa2a00a3e30 0x7fa2a00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:24.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.792+0000 7fa2b3e00640 1 -- 172.21.15.94:0/953387880 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fa2a40d0f90 con 0x7fa2bc102060 2024-04-25T15:02:24.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.792+0000 7fa2baa00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa2a00a3e30 0x7fa2a00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:24.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.792+0000 7fa2baa00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa2a00a3e30 0x7fa2a00a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac005e30 tx=0x7fa2ac009360 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:24.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.794+0000 7fa2b3e00640 1 -- 172.21.15.94:0/953387880 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2a4053ff0 con 0x7fa2bc102060 2024-04-25T15:02:24.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.998+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2bc10a360 con 0x7fa2bc102060 2024-04-25T15:02:25.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:24.999+0000 7fa2b3e00640 1 -- 172.21.15.94:0/953387880 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fa2a409d470 con 0x7fa2bc102060 2024-04-25T15:02:25.000 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:02:25.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa2a00a3e30 msgr2=0x7fa2a00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa2a00a3e30 0x7fa2a00a62f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac005e30 tx=0x7fa2ac009360 comp rx=0 tx=0).stop 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102060 msgr2=0x7fa2bc1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102060 0x7fa2bc1986a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9f0 tx=0x7fa2a400edf0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 shutdown_connections 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa2a00a3e30 0x7fa2a00a62f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2bc19cf60 0x7fa2bc19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2bc107130 0x7fa2bc198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 --2- 172.21.15.94:0/953387880 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa2bc102060 0x7fa2bc1986a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 >> 172.21.15.94:0/953387880 conn(0x7fa2bc0fd920 msgr2=0x7fa2bc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 shutdown_connections 2024-04-25T15:02:25.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.001+0000 7fa2c0e00640 1 -- 172.21.15.94:0/953387880 wait complete. 2024-04-25T15:02:25.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:02:25.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 -- 172.21.15.94:0/2987263929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e81043a0 msgr2=0x7f86e8104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 --2- 172.21.15.94:0/2987263929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e81043a0 0x7f86e8104780 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009900 tx=0x7f86dc02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 -- 172.21.15.94:0/2987263929 shutdown_connections 2024-04-25T15:02:25.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 --2- 172.21.15.94:0/2987263929 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86e8109470 0x7f86e8111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 --2- 172.21.15.94:0/2987263929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e8104d50 0x7f86e8108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 --2- 172.21.15.94:0/2987263929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e81043a0 0x7f86e8104780 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 -- 172.21.15.94:0/2987263929 >> 172.21.15.94:0/2987263929 conn(0x7f86e80ffc00 msgr2=0x7f86e8102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.213+0000 7f86ef800640 1 -- 172.21.15.94:0/2987263929 shutdown_connections 2024-04-25T15:02:25.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.214+0000 7f86ef800640 1 -- 172.21.15.94:0/2987263929 wait complete. 2024-04-25T15:02:25.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.214+0000 7f86ef800640 1 Processor -- start 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.214+0000 7f86ef800640 1 -- start start 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ef800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86e81043a0 0x7f86e81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ef800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e8109470 0x7f86e81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ef800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e819a7c0 0x7f86e819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ef800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f86e8114420 con 0x7f86e819a7c0 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ef800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81142a0 con 0x7f86e81043a0 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ef800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86e81145a0 con 0x7f86e8109470 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ed800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e819a7c0 0x7f86e819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ed800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e819a7c0 0x7f86e819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52432/0 (socket says 172.21.15.94:52432) 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ed800640 1 -- 172.21.15.94:0/3288279747 learned_addr learned my addr 172.21.15.94:0/3288279747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86ece00640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86e81043a0 0x7f86e81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.215+0000 7f86e7e00640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e8109470 0x7f86e81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ed800640 1 -- 172.21.15.94:0/3288279747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e8109470 msgr2=0x7f86e81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ed800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e8109470 0x7f86e81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ed800640 1 -- 172.21.15.94:0/3288279747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86e81043a0 msgr2=0x7f86e81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ed800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86e81043a0 0x7f86e81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ed800640 1 -- 172.21.15.94:0/3288279747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e819b350 con 0x7f86e819a7c0 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ed800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e819a7c0 0x7f86e819ac70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86e000eb10 tx=0x7f86e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86e5600640 1 -- 172.21.15.94:0/3288279747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0004b20 con 0x7f86e819a7c0 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86e5600640 1 -- 172.21.15.94:0/3288279747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e0004cc0 con 0x7f86e819a7c0 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e819b640 con 0x7f86e819a7c0 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86e5600640 1 -- 172.21.15.94:0/3288279747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0010670 con 0x7f86e819a7c0 2024-04-25T15:02:25.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.216+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e81a77d0 con 0x7f86e819a7c0 2024-04-25T15:02:25.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.218+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e8105870 con 0x7f86e819a7c0 2024-04-25T15:02:25.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.220+0000 7f86e5600640 1 -- 172.21.15.94:0/3288279747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f86e0005020 con 0x7f86e819a7c0 2024-04-25T15:02:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.221+0000 7f86e5600640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f86d00a3db0 0x7f86d00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.221+0000 7f86e5600640 1 -- 172.21.15.94:0/3288279747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f86e00d0ef0 con 0x7f86e819a7c0 2024-04-25T15:02:25.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.221+0000 7f86ece00640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f86d00a3db0 0x7f86d00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.222+0000 7f86ece00640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f86d00a3db0 0x7f86d00a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005ab0 tx=0x7f86dc005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.225+0000 7f86e5600640 1 -- 172.21.15.94:0/3288279747 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f86e819b640 con 0x7f86e819a7c0 2024-04-25T15:02:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 -- 172.21.15.94:0/1996113922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03741043a0 msgr2=0x7f0374104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 --2- 172.21.15.94:0/1996113922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03741043a0 0x7f0374104780 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0364009900 tx=0x7f036402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 -- 172.21.15.94:0/1996113922 shutdown_connections 2024-04-25T15:02:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 --2- 172.21.15.94:0/1996113922 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f0374109470 0x7f0374111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 --2- 172.21.15.94:0/1996113922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0374104d50 0x7f0374108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 --2- 172.21.15.94:0/1996113922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f03741043a0 0x7f0374104780 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 -- 172.21.15.94:0/1996113922 >> 172.21.15.94:0/1996113922 conn(0x7f03740ffc40 msgr2=0x7f0374102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 -- 172.21.15.94:0/1996113922 shutdown_connections 2024-04-25T15:02:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.417+0000 7f0379200640 1 -- 172.21.15.94:0/1996113922 wait complete. 2024-04-25T15:02:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.419+0000 7f0379200640 1 Processor -- start 2024-04-25T15:02:25.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.419+0000 7f0379200640 1 -- start start 2024-04-25T15:02:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0379200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03741043a0 0x7f03741a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0379200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0374109470 0x7f03741a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0379200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f037419a6f0 0x7f037419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0379200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03741143d0 con 0x7f037419a6f0 2024-04-25T15:02:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0379200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f0374114250 con 0x7f03741043a0 2024-04-25T15:02:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0379200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0374114550 con 0x7f0374109470 2024-04-25T15:02:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0371e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03741043a0 0x7f03741a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0372800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f037419a6f0 0x7f037419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0371e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03741043a0 0x7f03741a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:43316/0 (socket says 172.21.15.94:43316) 2024-04-25T15:02:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f0371e00640 1 -- 172.21.15.94:0/3992478944 learned_addr learned my addr 172.21.15.94:0/3992478944 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.420+0000 7f036be00640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0374109470 0x7f03741a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0371e00640 1 -- 172.21.15.94:0/3992478944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0374109470 msgr2=0x7f03741a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0371e00640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0374109470 0x7f03741a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0371e00640 1 -- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f037419a6f0 msgr2=0x7f037419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0371e00640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f037419a6f0 0x7f037419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0371e00640 1 -- 172.21.15.94:0/3992478944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037419b2f0 con 0x7f03741043a0 2024-04-25T15:02:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0371e00640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03741043a0 0x7f03741a05e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0364005940 tx=0x7f03640040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f036aa00640 1 -- 172.21.15.94:0/3992478944 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036403f030 con 0x7f03741043a0 2024-04-25T15:02:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f037419b580 con 0x7f03741043a0 2024-04-25T15:02:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f036aa00640 1 -- 172.21.15.94:0/3992478944 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03640044c0 con 0x7f03741043a0 2024-04-25T15:02:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f036aa00640 1 -- 172.21.15.94:0/3992478944 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364035070 con 0x7f03741043a0 2024-04-25T15:02:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.421+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03741a76c0 con 0x7f03741043a0 2024-04-25T15:02:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.425+0000 7f037bf87740 1 Processor -- start 2024-04-25T15:02:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f037bf87740 1 -- start start 2024-04-25T15:02:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f037bf87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f45f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f037bf87740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f45fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f037bf87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f45ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f037bf87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9f4558900 con 0x55f9f45f2820 2024-04-25T15:02:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f037bf87740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f9f44feef0 con 0x55f9f45f3140 2024-04-25T15:02:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f037bf87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9f45548c0 con 0x55f9f45fda00 2024-04-25T15:02:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f0372800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f45f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f0372800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f45f2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52464/0 (socket says 172.21.15.94:52464) 2024-04-25T15:02:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f0372800640 1 -- 172.21.15.94:0/1797440870 learned_addr learned my addr 172.21.15.94:0/1797440870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.426+0000 7f036be00640 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f45ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f036aa00640 1 -- 172.21.15.94:0/3992478944 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f0364042030 con 0x7f03741043a0 2024-04-25T15:02:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0372800640 1 -- 172.21.15.94:0/1797440870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 msgr2=0x55f9f45ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:02:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0372800640 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f45ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0372800640 1 -- 172.21.15.94:0/1797440870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 msgr2=0x55f9f45fd4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T15:02:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0372800640 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f45fd4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0372800640 1 -- 172.21.15.94:0/1797440870 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f426b210 con 0x55f9f45f2820 2024-04-25T15:02:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0372800640 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f45f2c00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f036c009f50 tx=0x7f036c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c4ab76cc99dda5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0363400640 1 -- 172.21.15.94:0/1797440870 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c004d20 con 0x55f9f45f2820 2024-04-25T15:02:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.427+0000 7f0363400640 1 -- 172.21.15.94:0/1797440870 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036c004050 con 0x55f9f45f2820 2024-04-25T15:02:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f0363400640 1 -- 172.21.15.94:0/1797440870 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c005280 con 0x55f9f45f2820 2024-04-25T15:02:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f037bf87740 1 -- 172.21.15.94:0/1797440870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 msgr2=0x55f9f45f2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f037bf87740 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f45f2c00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f036c009f50 tx=0x7f036c0118d0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f036aa00640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f03580a3ee0 0x7f03580a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f037bf87740 1 -- 172.21.15.94:0/1797440870 shutdown_connections 2024-04-25T15:02:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f037bf87740 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f45ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f037bf87740 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f45fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f037bf87740 1 --2- 172.21.15.94:0/1797440870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f45f2c00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.428+0000 7f037bf87740 1 -- 172.21.15.94:0/1797440870 >> 172.21.15.94:0/1797440870 conn(0x55f9f45eed30 msgr2=0x55f9f45ef110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.429+0000 7f036aa00640 1 -- 172.21.15.94:0/3992478944 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f03640f08f0 con 0x7f03741043a0 2024-04-25T15:02:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.429+0000 7f0371e00640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f03580a3ee0 0x7f03580a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.429+0000 7f037bf87740 1 -- 172.21.15.94:0/1797440870 shutdown_connections 2024-04-25T15:02:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.429+0000 7f037bf87740 1 -- 172.21.15.94:0/1797440870 wait complete. 2024-04-25T15:02:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.429+0000 7f037bf87740 1 -- start start 2024-04-25T15:02:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.429+0000 7f0371e00640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f03580a3ee0 0x7f03580a63a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03640edf90 tx=0x7f0364009560 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.429+0000 7f037bf87740 1 Processor -- start 2024-04-25T15:02:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f037bf87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f45fa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f037bf87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f3140 0x55f9f45fadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f037bf87740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45fda00 0x55f9f45fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f037bf87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x55f9f4600630 con 0x55f9f45f3140 2024-04-25T15:02:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f037bf87740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x55f9f4512310 con 0x55f9f45fda00 2024-04-25T15:02:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f037bf87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f9f46007b0 con 0x55f9f45f2820 2024-04-25T15:02:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f037bf87740 10 client.0 _ll_register_callbacks cb 0x55f9f44e0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:02:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f036be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f3140 0x55f9f45fadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f036be00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f3140 0x55f9f45fadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52468/0 (socket says 172.21.15.94:52468) 2024-04-25T15:02:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f036be00640 1 -- 172.21.15.94:0/791972657 learned_addr learned my addr 172.21.15.94:0/791972657 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f0372800640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f45fa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.430+0000 7f0372800640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45fda00 0x55f9f45fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f036be00640 1 -- 172.21.15.94:0/791972657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 msgr2=0x55f9f45fa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:02:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f036be00640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f45fa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f036be00640 1 -- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45fda00 msgr2=0x55f9f45fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:02:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f036be00640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45fda00 0x55f9f45fb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f036be00640 1 -- 172.21.15.94:0/791972657 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f4337620 con 0x55f9f45f3140 2024-04-25T15:02:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f0372800640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f45fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:02:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f036be00640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f3140 0x55f9f45fadf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f035c012700 tx=0x7f035c012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c033d67b15c04632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f0357e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T15:02:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f0357e00640 1 -- 172.21.15.94:0/791972657 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c004020 con 0x55f9f45f3140 2024-04-25T15:02:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f0357e00640 1 -- 172.21.15.94:0/791972657 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f035c0041c0 con 0x55f9f45f3140 2024-04-25T15:02:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.431+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f036c004050 con 0x55f9f45f3140 2024-04-25T15:02:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f0357e00640 1 -- 172.21.15.94:0/791972657 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f035c0043c0 con 0x55f9f45f3140 2024-04-25T15:02:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 10 client.15168 fetch_fsmap learned FSMap version 19 2024-04-25T15:02:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f9f460c810 con 0x55f9f45f3140 2024-04-25T15:02:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f0357e00640 1 -- 172.21.15.94:0/791972657 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036c004050 con 0x55f9f45f3140 2024-04-25T15:02:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f0357e00640 1 -- 172.21.15.94:0/791972657 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f035c003ae0 con 0x55f9f45f3140 2024-04-25T15:02:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 10 client.15168 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T15:02:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 10 client.15168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T15:02:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 20 client.15168 populate_metadata read hostname 'smithi094' 2024-04-25T15:02:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f460dd20 0x55f9f4610130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 4 client.15168 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T15:02:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.432+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 --> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9f456c8b0 con 0x55f9f460dd20 2024-04-25T15:02:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.433+0000 7f0372800640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f460dd20 0x55f9f4610130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.433+0000 7f0372800640 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f460dd20 0x55f9f4610130 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5849f719ded51cad server_cookie=f4219e9ac5e4f65c in_seq=0 out_seq=0 2024-04-25T15:02:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.433+0000 7f0357e00640 10 client.15168 ms_handle_connect on v2:172.21.15.137:6838/3444396688 2024-04-25T15:02:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f0357e00640 1 -- 172.21.15.94:0/791972657 <== mds.0 v2:172.21.15.137:6838/3444396688 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9f456c8b0 con 0x55f9f460dd20 2024-04-25T15:02:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f0357e00640 10 client.15168 handle_command_reply: tid=0 2024-04-25T15:02:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 client.15168 shutdown 2024-04-25T15:02:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f3140 msgr2=0x55f9f45fadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f3140 0x55f9f45fadf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f035c012700 tx=0x7f035c012bc0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 shutdown_connections 2024-04-25T15:02:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45fda00 0x55f9f45fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f3140 0x55f9f45fadf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f460dd20 0x55f9f4610130 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 --2- 172.21.15.94:0/791972657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f45fa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 >> 172.21.15.94:0/791972657 conn(0x55f9f45eed30 msgr2=0x55f9f45f4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f0357e00640 0 client.15168 ms_handle_reset on v2:172.21.15.137:6838/3444396688 2024-04-25T15:02:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 shutdown_connections 2024-04-25T15:02:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.434+0000 7f037bf87740 1 -- 172.21.15.94:0/791972657 wait complete. 2024-04-25T15:02:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.435+0000 7f037bf87740 20 client.15168 trim_cache size 0 max 16384 2024-04-25T15:02:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 Processor -- start 2024-04-25T15:02:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 -- start start 2024-04-25T15:02:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f46094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f45f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45fda00 0x55f9f45f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03741adcb0 con 0x55f9f45fda00 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03741b0530 con 0x55f9f45f3140 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f037bf87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03741ae320 con 0x55f9f45f2820 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f0372800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45fda00 0x55f9f45f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f0372800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45fda00 0x55f9f45f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52478/0 (socket says 172.21.15.94:52478) 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f0372800640 1 -- 172.21.15.94:0/179673573 learned_addr learned my addr 172.21.15.94:0/179673573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f036be00640 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f45f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.451+0000 7f0372800640 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f46094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.452+0000 7f0372800640 1 -- 172.21.15.94:0/179673573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 msgr2=0x55f9f46094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.452+0000 7f0372800640 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f46094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.452+0000 7f0372800640 1 -- 172.21.15.94:0/179673573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 msgr2=0x55f9f45f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:02:25.464 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.452+0000 7f0372800640 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f45f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.452+0000 7f0372800640 1 -- 172.21.15.94:0/179673573 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f45f5720 con 0x55f9f45fda00 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.452+0000 7f0372800640 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45fda00 0x55f9f45f51e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f036c0048f0 tx=0x7f036c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e73f2bb82de540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.452+0000 7f0362a00640 1 -- 172.21.15.94:0/179673573 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c003ce0 con 0x55f9f45fda00 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f0362a00640 1 -- 172.21.15.94:0/179673573 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f036c004e40 con 0x55f9f45fda00 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f0362a00640 1 -- 172.21.15.94:0/179673573 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c005940 con 0x55f9f45fda00 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 -- 172.21.15.94:0/179673573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45fda00 msgr2=0x55f9f45f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45fda00 0x55f9f45f51e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f036c0048f0 tx=0x7f036c0118d0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 -- 172.21.15.94:0/179673573 shutdown_connections 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45fda00 0x55f9f45f51e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f45f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 --2- 172.21.15.94:0/179673573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45f2820 0x55f9f46094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 -- 172.21.15.94:0/179673573 >> 172.21.15.94:0/179673573 conn(0x55f9f45eed30 msgr2=0x55f9f45fa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 -- 172.21.15.94:0/179673573 shutdown_connections 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.453+0000 7f037bf87740 1 -- 172.21.15.94:0/179673573 wait complete. 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 -- start start 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 Processor -- start 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f4674e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f4675370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f4602e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f03741ae9f0 con 0x55f9f45f2820 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f03741afe60 con 0x55f9f45f3140 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.454+0000 7f037bf87740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03741af0c0 con 0x55f9f45fda00 2024-04-25T15:02:25.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f037bf87740 10 client.0 _ll_register_callbacks cb 0x7f035c009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f4674e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f4674e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52480/0 (socket says 172.21.15.94:52480) 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 -- 172.21.15.94:0/123800807 learned_addr learned my addr 172.21.15.94:0/123800807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f036be00640 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f4675370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f4602e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 -- 172.21.15.94:0/123800807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 msgr2=0x55f9f4602e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f4602e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 -- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 msgr2=0x55f9f4675370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f4675370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.455+0000 7f0372800640 1 -- 172.21.15.94:0/123800807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f46034c0 con 0x55f9f45f2820 2024-04-25T15:02:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f0372800640 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f4674e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f036c0044a0 tx=0x7f036c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4bbb0ccc9c1a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f0357e00640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f0357e00640 1 -- 172.21.15.94:0/123800807 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036c003b60 con 0x55f9f45f2820 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9f4566320 con 0x55f9f45f2820 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f0357e00640 1 -- 172.21.15.94:0/123800807 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f036c003d00 con 0x55f9f45f2820 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f0357e00640 1 -- 172.21.15.94:0/123800807 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f036c004920 con 0x55f9f45f2820 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f037bf87740 10 client.15174 fetch_fsmap learned FSMap version 19 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f9f467c600 con 0x55f9f45f2820 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.456+0000 7f0357e00640 1 -- 172.21.15.94:0/123800807 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f9f4566320 con 0x55f9f45f2820 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f0357e00640 1 -- 172.21.15.94:0/123800807 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f036c025040 con 0x55f9f45f2820 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f037bf87740 10 client.15174 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f037bf87740 10 client.15174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T15:02:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f037bf87740 20 client.15174 populate_metadata read hostname 'smithi094' 2024-04-25T15:02:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f037bf87740 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f467cb90 0x55f9f467efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f037bf87740 4 client.15174 mds_command: new command op to 24596 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T15:02:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 --> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9f4560190 con 0x55f9f467cb90 2024-04-25T15:02:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.457+0000 7f036be00640 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f467cb90 0x55f9f467efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.458+0000 7f036be00640 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f467cb90 0x55f9f467efb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f002a0f8b92579a8 server_cookie=42407823a41e2f51 in_seq=0 out_seq=0 2024-04-25T15:02:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.458+0000 7f0357e00640 10 client.15174 ms_handle_connect on v2:172.21.15.137:6838/3444396688 2024-04-25T15:02:25.469 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:25 smithi094 ceph-mon[27862]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:02:25.469 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:25 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2676169984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:02:25.469 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:25 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/953387880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:02:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.470+0000 7f0357e00640 1 -- 172.21.15.94:0/123800807 <== mds.0 v2:172.21.15.137:6838/3444396688 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9f4560190 con 0x55f9f467cb90 2024-04-25T15:02:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.470+0000 7f0357e00640 10 client.15174 handle_command_reply: tid=0 2024-04-25T15:02:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.470+0000 7f037bf87740 1 client.15174 shutdown 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 msgr2=0x55f9f4674e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f4674e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f036c0044a0 tx=0x7f036c009520 comp rx=0 tx=0).stop 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 shutdown_connections 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:6838/3444396688,v1:172.21.15.137:6839/3444396688] conn(0x55f9f467cb90 0x55f9f467efb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f9f45fda00 0x55f9f4602e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x55f9f45f3140 0x55f9f4675370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 --2- 172.21.15.94:0/123800807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f9f45f2820 0x55f9f4674e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f0357e00640 0 client.15174 ms_handle_reset on v2:172.21.15.137:6838/3444396688 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 >> 172.21.15.94:0/123800807 conn(0x55f9f45eed30 msgr2=0x55f9f45fb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 shutdown_connections 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 1 -- 172.21.15.94:0/123800807 wait complete. 2024-04-25T15:02:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.471+0000 7f037bf87740 20 client.15174 trim_cache size 0 max 16384 2024-04-25T15:02:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-04-25T15:02:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-04-25T15:02:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-04-25T15:02:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-04-25T15:02:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f03580a3ee0 msgr2=0x7f03580a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f03580a3ee0 0x7f03580a63a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f03640edf90 tx=0x7f0364009560 comp rx=0 tx=0).stop 2024-04-25T15:02:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03741043a0 msgr2=0x7f03741a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03741043a0 0x7f03741a05e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0364005940 tx=0x7f03640040c0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 shutdown_connections 2024-04-25T15:02:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f03580a3ee0 0x7f03580a63a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f037419a6f0 0x7f037419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0374109470 0x7f03741a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 --2- 172.21.15.94:0/3992478944 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f03741043a0 0x7f03741a05e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 >> 172.21.15.94:0/3992478944 conn(0x7f03740ffc40 msgr2=0x7f0374101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 shutdown_connections 2024-04-25T15:02:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-04-25T15:02:25.472+0000 7f0379200640 1 -- 172.21.15.94:0/3992478944 wait complete. 2024-04-25T15:02:25.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.486+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86e8068d70 con 0x7f86e819a7c0 2024-04-25T15:02:25.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.487+0000 7f86e5600640 1 -- 172.21.15.94:0/3288279747 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f86e8068d70 con 0x7f86e819a7c0 2024-04-25T15:02:25.489 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:25.489 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:02:25.489 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f86d00a3db0 msgr2=0x7f86d00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f86d00a3db0 0x7f86d00a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f86dc005ab0 tx=0x7f86dc005a20 comp rx=0 tx=0).stop 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e819a7c0 msgr2=0x7f86e819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e819a7c0 0x7f86e819ac70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86e000eb10 tx=0x7f86e000c2d0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 shutdown_connections 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f86d00a3db0 0x7f86d00a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86e819a7c0 0x7f86e819ac70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86e8109470 0x7f86e81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 --2- 172.21.15.94:0/3288279747 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f86e81043a0 0x7f86e81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 >> 172.21.15.94:0/3288279747 conn(0x7f86e80ffc00 msgr2=0x7f86e8100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 shutdown_connections 2024-04-25T15:02:25.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.490+0000 7f86ef800640 1 -- 172.21.15.94:0/3288279747 wait complete. 2024-04-25T15:02:25.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 -- 172.21.15.94:0/948544593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108f40 msgr2=0x7f67f8115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 --2- 172.21.15.94:0/948544593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108f40 0x7f67f8115400 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009c10 tx=0x7f67f002a6f0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 -- 172.21.15.94:0/948544593 shutdown_connections 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 --2- 172.21.15.94:0/948544593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108f40 0x7f67f8115400 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 --2- 172.21.15.94:0/948544593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108580 0x7f67f8108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 --2- 172.21.15.94:0/948544593 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f810e640 0x7f67f810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 -- 172.21.15.94:0/948544593 >> 172.21.15.94:0/948544593 conn(0x7f67f8075750 msgr2=0x7f67f8106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 -- 172.21.15.94:0/948544593 shutdown_connections 2024-04-25T15:02:25.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.695+0000 7f67ffe00640 1 -- 172.21.15.94:0/948544593 wait complete. 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.696+0000 7f67ffe00640 1 Processor -- start 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.696+0000 7f67ffe00640 1 -- start start 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108580 0x7f67f81a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67ffe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108f40 0x7f67f81a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f810e640 0x7f67f819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67ffe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f67f81187b0 con 0x7f67f8108f40 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67ffe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f67f8118630 con 0x7f67f810e640 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67ffe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67f8118930 con 0x7f67f8108580 2024-04-25T15:02:25.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67fca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108f40 0x7f67f81a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67fde00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f810e640 0x7f67f819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67fca00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108f40 0x7f67f81a4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52498/0 (socket says 172.21.15.94:52498) 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67fca00640 1 -- 172.21.15.94:0/2530287656 learned_addr learned my addr 172.21.15.94:0/2530287656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67fd400640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108580 0x7f67f81a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67fde00640 1 -- 172.21.15.94:0/2530287656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108580 msgr2=0x7f67f81a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.697+0000 7f67fde00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108580 0x7f67f81a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67fde00640 1 -- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108f40 msgr2=0x7f67f81a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67fde00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108f40 0x7f67f81a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67fde00640 1 -- 172.21.15.94:0/2530287656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f819f250 con 0x7f67f810e640 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67fca00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108f40 0x7f67f81a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:02:25.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67fde00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f810e640 0x7f67f819ead0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f67f0034040 tx=0x7f67f0003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67f6000640 1 -- 172.21.15.94:0/2530287656 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0041020 con 0x7f67f810e640 2024-04-25T15:02:25.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f819f4e0 con 0x7f67f810e640 2024-04-25T15:02:25.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67f6000640 1 -- 172.21.15.94:0/2530287656 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f00318f0 con 0x7f67f810e640 2024-04-25T15:02:25.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67f6000640 1 -- 172.21.15.94:0/2530287656 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0036040 con 0x7f67f810e640 2024-04-25T15:02:25.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.698+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f81ab7e0 con 0x7f67f810e640 2024-04-25T15:02:25.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.699+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f80691c0 con 0x7f67f810e640 2024-04-25T15:02:25.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.702+0000 7f67f6000640 1 -- 172.21.15.94:0/2530287656 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f67f0044020 con 0x7f67f810e640 2024-04-25T15:02:25.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.703+0000 7f67f6000640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f67e00a3db0 0x7f67e00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:25.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.703+0000 7f67f6000640 1 -- 172.21.15.94:0/2530287656 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f67f00f1960 con 0x7f67f810e640 2024-04-25T15:02:25.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.703+0000 7f67fd400640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f67e00a3db0 0x7f67e00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:25.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.704+0000 7f67fd400640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f67e00a3db0 0x7f67e00a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0095f0 tx=0x7f67ec005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:25.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:25 smithi138 ceph-mon[32875]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:02:25.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:25 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2676169984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:02:25.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:25 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/953387880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:02:25.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.705+0000 7f67f6000640 1 -- 172.21.15.94:0/2530287656 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67f004b020 con 0x7f67f810e640 2024-04-25T15:02:25.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:25 smithi137 ceph-mon[32439]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:02:25.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:25 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2676169984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:02:25.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:25 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/953387880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:02:25.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.910+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67f81109c0 con 0x7f67f810e640 2024-04-25T15:02:25.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.912+0000 7f67f6000640 1 -- 172.21.15.94:0/2530287656 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23061 (secure 0 0 0) 0x7f67f00bde40 con 0x7f67f810e640 2024-04-25T15:02:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:25.914 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":89,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:02:09.415151+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/702744035":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3354212797":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6801/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1994260887":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/2523030980":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6800/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:25.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f67e00a3db0 msgr2=0x7f67e00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f67e00a3db0 0x7f67e00a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0095f0 tx=0x7f67ec005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f810e640 msgr2=0x7f67f819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f810e640 0x7f67f819ead0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f67f0034040 tx=0x7f67f0003b80 comp rx=0 tx=0).stop 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 shutdown_connections 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f67e00a3db0 0x7f67e00a6270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f67f810e640 0x7f67f819ead0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67f8108f40 0x7f67f81a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 --2- 172.21.15.94:0/2530287656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67f8108580 0x7f67f81a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 >> 172.21.15.94:0/2530287656 conn(0x7f67f8075750 msgr2=0x7f67f80768a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 shutdown_connections 2024-04-25T15:02:25.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:25.916+0000 7f67ffe00640 1 -- 172.21.15.94:0/2530287656 wait complete. 2024-04-25T15:02:25.937 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:02:25.937 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:02:26.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.120+0000 7f9361800640 1 -- 172.21.15.94:0/2980881281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c107140 msgr2=0x7f935c10ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.120+0000 7f9361800640 1 --2- 172.21.15.94:0/2980881281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c107140 0x7f935c10ed10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9354009be0 tx=0x7f935402a6c0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.120+0000 7f9361800640 1 -- 172.21.15.94:0/2980881281 shutdown_connections 2024-04-25T15:02:26.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.120+0000 7f9361800640 1 --2- 172.21.15.94:0/2980881281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c107140 0x7f935c10ed10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.120+0000 7f9361800640 1 --2- 172.21.15.94:0/2980881281 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c102a20 0x7f935c106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.120+0000 7f9361800640 1 --2- 172.21.15.94:0/2980881281 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f935c102070 0x7f935c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.120+0000 7f9361800640 1 -- 172.21.15.94:0/2980881281 >> 172.21.15.94:0/2980881281 conn(0x7f935c0fd8f0 msgr2=0x7f935c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:26.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.121+0000 7f9361800640 1 -- 172.21.15.94:0/2980881281 shutdown_connections 2024-04-25T15:02:26.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.121+0000 7f9361800640 1 -- 172.21.15.94:0/2980881281 wait complete. 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.121+0000 7f9361800640 1 Processor -- start 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9361800640 1 -- start start 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9361800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c102070 0x7f935c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9361800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f935c107140 0x7f935c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9361800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c19a5f0 0x7f935c19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9361800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f935c1120b0 con 0x7f935c19a5f0 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9361800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f935c111f30 con 0x7f935c107140 2024-04-25T15:02:26.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9361800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f935c112230 con 0x7f935c102070 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9360e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c19a5f0 0x7f935c19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9360e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c19a5f0 0x7f935c19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52516/0 (socket says 172.21.15.94:52516) 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9360e00640 1 -- 172.21.15.94:0/1525370106 learned_addr learned my addr 172.21.15.94:0/1525370106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f935be00640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c102070 0x7f935c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.122+0000 7f9353e00640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f935c107140 0x7f935c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9360e00640 1 -- 172.21.15.94:0/1525370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c102070 msgr2=0x7f935c1a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9360e00640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c102070 0x7f935c1a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9360e00640 1 -- 172.21.15.94:0/1525370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f935c107140 msgr2=0x7f935c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9360e00640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f935c107140 0x7f935c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9360e00640 1 -- 172.21.15.94:0/1525370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c19b110 con 0x7f935c19a5f0 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9360e00640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c19a5f0 0x7f935c19aaa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9354009b30 tx=0x7f9354003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9359600640 1 -- 172.21.15.94:0/1525370106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354040020 con 0x7f935c19a5f0 2024-04-25T15:02:26.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9359600640 1 -- 172.21.15.94:0/1525370106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93540318f0 con 0x7f935c19a5f0 2024-04-25T15:02:26.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c19b3a0 con 0x7f935c19a5f0 2024-04-25T15:02:26.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9359600640 1 -- 172.21.15.94:0/1525370106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354036040 con 0x7f935c19a5f0 2024-04-25T15:02:26.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.123+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c1a7590 con 0x7f935c19a5f0 2024-04-25T15:02:26.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.125+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c0690e0 con 0x7f935c19a5f0 2024-04-25T15:02:26.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.126+0000 7f9359600640 1 -- 172.21.15.94:0/1525370106 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f9354043020 con 0x7f935c19a5f0 2024-04-25T15:02:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.129+0000 7f9359600640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f93400a3db0 0x7f93400a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.129+0000 7f9359600640 1 -- 172.21.15.94:0/1525370106 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f93540f1570 con 0x7f935c19a5f0 2024-04-25T15:02:26.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.129+0000 7f935be00640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f93400a3db0 0x7f93400a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.129+0000 7f935be00640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f93400a3db0 0x7f93400a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c009260 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:26.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.133+0000 7f9359600640 1 -- 172.21.15.94:0/1525370106 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f935404a020 con 0x7f935c19a5f0 2024-04-25T15:02:26.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.340+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f935c102b20 con 0x7f935c19a5f0 2024-04-25T15:02:26.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.341+0000 7f9359600640 1 -- 172.21.15.94:0/1525370106 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f93540bda50 con 0x7f935c19a5f0 2024-04-25T15:02:26.343 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:02:26.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f93400a3db0 msgr2=0x7f93400a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f93400a3db0 0x7f93400a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f934c005fd0 tx=0x7f934c009260 comp rx=0 tx=0).stop 2024-04-25T15:02:26.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c19a5f0 msgr2=0x7f935c19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c19a5f0 0x7f935c19aaa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9354009b30 tx=0x7f9354003c10 comp rx=0 tx=0).stop 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 shutdown_connections 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f93400a3db0 0x7f93400a6270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f935c19a5f0 0x7f935c19aaa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f935c107140 0x7f935c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 --2- 172.21.15.94:0/1525370106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f935c102070 0x7f935c1a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 >> 172.21.15.94:0/1525370106 conn(0x7f935c0fd8f0 msgr2=0x7f935c0ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 shutdown_connections 2024-04-25T15:02:26.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.344+0000 7f9361800640 1 -- 172.21.15.94:0/1525370106 wait complete. 2024-04-25T15:02:26.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 -- 172.21.15.94:0/2768355892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 msgr2=0x7f546010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 --2- 172.21.15.94:0/2768355892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f546010bdc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5448009900 tx=0x7f544802a480 comp rx=0 tx=0).stop 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 -- 172.21.15.94:0/2768355892 shutdown_connections 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 --2- 172.21.15.94:0/2768355892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f546010c3e0 0x7f546010e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 --2- 172.21.15.94:0/2768355892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f546010bdc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 --2- 172.21.15.94:0/2768355892 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460102fc0 0x7f54601033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 -- 172.21.15.94:0/2768355892 >> 172.21.15.94:0/2768355892 conn(0x7f54600fc7a0 msgr2=0x7f54600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:26.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 -- 172.21.15.94:0/2768355892 shutdown_connections 2024-04-25T15:02:26.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.564+0000 7f5465c00640 1 -- 172.21.15.94:0/2768355892 wait complete. 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.565+0000 7f5465c00640 1 Processor -- start 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.565+0000 7f5465c00640 1 -- start start 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f5465c00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460102fc0 0x7f5460198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f5465c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f5460198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f5465c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f546010c3e0 0x7f546019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f5465c00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f54601120a0 con 0x7f54601038e0 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f5465c00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5460111f20 con 0x7f5460102fc0 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f5465c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5460112220 con 0x7f546010c3e0 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f5460198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f5460198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52538/0 (socket says 172.21.15.94:52538) 2024-04-25T15:02:26.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 -- 172.21.15.94:0/3083660579 learned_addr learned my addr 172.21.15.94:0/3083660579 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f5465200640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f546010c3e0 0x7f546019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545fe00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460102fc0 0x7f5460198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 -- 172.21.15.94:0/3083660579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f546010c3e0 msgr2=0x7f546019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f546010c3e0 0x7f546019cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 -- 172.21.15.94:0/3083660579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460102fc0 msgr2=0x7f5460198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460102fc0 0x7f5460198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.566+0000 7f545f400640 1 -- 172.21.15.94:0/3083660579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546019d630 con 0x7f54601038e0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.567+0000 7f545f400640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f5460198b80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5448005ab0 tx=0x7f5448003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.567+0000 7f545cc00640 1 -- 172.21.15.94:0/3083660579 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544803f070 con 0x7f54601038e0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.567+0000 7f545cc00640 1 -- 172.21.15.94:0/3083660579 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5448031b50 con 0x7f54601038e0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.567+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546019d8c0 con 0x7f54601038e0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.567+0000 7f545cc00640 1 -- 172.21.15.94:0/3083660579 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448035030 con 0x7f54601038e0 2024-04-25T15:02:26.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.567+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54601a51b0 con 0x7f54601038e0 2024-04-25T15:02:26.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.568+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5460068400 con 0x7f54601038e0 2024-04-25T15:02:26.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.571+0000 7f545cc00640 1 -- 172.21.15.94:0/3083660579 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f5448042070 con 0x7f54601038e0 2024-04-25T15:02:26.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.572+0000 7f545cc00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f54400a3db0 0x7f54400a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.572+0000 7f545cc00640 1 -- 172.21.15.94:0/3083660579 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f54480f0c40 con 0x7f54601038e0 2024-04-25T15:02:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.573+0000 7f545fe00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f54400a3db0 0x7f54400a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.573+0000 7f545fe00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f54400a3db0 0x7f54400a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5450009bd0 tx=0x7f5450005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:26.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.576+0000 7f545cc00640 1 -- 172.21.15.94:0/3083660579 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54480bcad0 con 0x7f54601038e0 2024-04-25T15:02:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:26 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3288279747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:26 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2530287656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:26.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:26 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3288279747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:26.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:26 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2530287656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:26.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:26 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3288279747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:26.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:26 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2530287656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:26.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.779+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5460107480 con 0x7f54601038e0 2024-04-25T15:02:26.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.780+0000 7f545cc00640 1 -- 172.21.15.94:0/3083660579 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f54480c0dd0 con 0x7f54601038e0 2024-04-25T15:02:26.782 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:02:26.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f54400a3db0 msgr2=0x7f54400a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f54400a3db0 0x7f54400a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5450009bd0 tx=0x7f5450005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 msgr2=0x7f5460198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f5460198b80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5448005ab0 tx=0x7f5448003b60 comp rx=0 tx=0).stop 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 shutdown_connections 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f54400a3db0 0x7f54400a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f546010c3e0 0x7f546019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f54601038e0 0x7f5460198b80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 --2- 172.21.15.94:0/3083660579 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f5460102fc0 0x7f5460198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 >> 172.21.15.94:0/3083660579 conn(0x7f54600fc7a0 msgr2=0x7f54600fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 shutdown_connections 2024-04-25T15:02:26.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.783+0000 7f5465c00640 1 -- 172.21.15.94:0/3083660579 wait complete. 2024-04-25T15:02:26.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:02:26.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.993+0000 7f7b4ee00640 1 -- 172.21.15.94:0/3249703677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104c10 msgr2=0x7f7b48108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.993+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/3249703677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104c10 0x7f7b48108c50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7b34009900 tx=0x7f7b3402a410 comp rx=0 tx=0).stop 2024-04-25T15:02:26.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.994+0000 7f7b4ee00640 1 -- 172.21.15.94:0/3249703677 shutdown_connections 2024-04-25T15:02:26.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.994+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/3249703677 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b48109330 0x7f7b48110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.994+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/3249703677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104c10 0x7f7b48108c50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.994+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/3249703677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b48104260 0x7f7b48104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.994+0000 7f7b4ee00640 1 -- 172.21.15.94:0/3249703677 >> 172.21.15.94:0/3249703677 conn(0x7f7b480ffa60 msgr2=0x7f7b48101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:26.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.994+0000 7f7b4ee00640 1 -- 172.21.15.94:0/3249703677 shutdown_connections 2024-04-25T15:02:26.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.994+0000 7f7b4ee00640 1 -- 172.21.15.94:0/3249703677 wait complete. 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 Processor -- start 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 -- start start 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104260 0x7f7b481a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b48109330 0x7f7b481a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4819a6a0 0x7f7b4819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481142c0 con 0x7f7b48104260 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48114140 con 0x7f7b48109330 2024-04-25T15:02:26.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48114440 con 0x7f7b4819a6a0 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.995+0000 7f7b4da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104260 0x7f7b481a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104260 0x7f7b481a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46298/0 (socket says 172.21.15.94:46298) 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 -- 172.21.15.94:0/2169498443 learned_addr learned my addr 172.21.15.94:0/2169498443 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4e400640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4819a6a0 0x7f7b4819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4d000640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b48109330 0x7f7b481a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 -- 172.21.15.94:0/2169498443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4819a6a0 msgr2=0x7f7b4819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4819a6a0 0x7f7b4819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 -- 172.21.15.94:0/2169498443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b48109330 msgr2=0x7f7b481a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b48109330 0x7f7b481a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 -- 172.21.15.94:0/2169498443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4819b2a0 con 0x7f7b48104260 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b4da00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104260 0x7f7b481a05e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e9f0 tx=0x7f7b3c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.996+0000 7f7b46000640 1 -- 172.21.15.94:0/2169498443 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00cd80 con 0x7f7b48104260 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.997+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b481a7380 con 0x7f7b48104260 2024-04-25T15:02:26.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.997+0000 7f7b46000640 1 -- 172.21.15.94:0/2169498443 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b3c0054d0 con 0x7f7b48104260 2024-04-25T15:02:27.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.997+0000 7f7b46000640 1 -- 172.21.15.94:0/2169498443 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c010690 con 0x7f7b48104260 2024-04-25T15:02:27.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.997+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b481a7730 con 0x7f7b48104260 2024-04-25T15:02:27.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.998+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b48105730 con 0x7f7b48104260 2024-04-25T15:02:27.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:26.999+0000 7f7b46000640 1 -- 172.21.15.94:0/2169498443 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7b3c004e90 con 0x7f7b48104260 2024-04-25T15:02:27.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.001+0000 7f7b46000640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7b300a3e30 0x7f7b300a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.001+0000 7f7b46000640 1 -- 172.21.15.94:0/2169498443 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7f7b3c0d0d60 con 0x7f7b48104260 2024-04-25T15:02:27.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.001+0000 7f7b4d000640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7b300a3e30 0x7f7b300a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.002+0000 7f7b4d000640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7b300a3e30 0x7f7b300a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b34031440 tx=0x7f7b34005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:27.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.005+0000 7f7b46000640 1 -- 172.21.15.94:0/2169498443 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b3c09d240 con 0x7f7b48104260 2024-04-25T15:02:27.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.192+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7b4810c560 con 0x7f7b48104260 2024-04-25T15:02:27.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.193+0000 7f7b46000640 1 -- 172.21.15.94:0/2169498443 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f7b3c09cb40 con 0x7f7b48104260 2024-04-25T15:02:27.194 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7b300a3e30 msgr2=0x7f7b300a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7b300a3e30 0x7f7b300a62f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7b34031440 tx=0x7f7b34005a20 comp rx=0 tx=0).stop 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104260 msgr2=0x7f7b481a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104260 0x7f7b481a05e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c00e9f0 tx=0x7f7b3c00edf0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 shutdown_connections 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7b300a3e30 0x7f7b300a62f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b4819a6a0 0x7f7b4819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7b48109330 0x7f7b481a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 --2- 172.21.15.94:0/2169498443 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b48104260 0x7f7b481a05e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.195+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 >> 172.21.15.94:0/2169498443 conn(0x7f7b480ffa60 msgr2=0x7f7b48100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:27.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.196+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 shutdown_connections 2024-04-25T15:02:27.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.196+0000 7f7b4ee00640 1 -- 172.21.15.94:0/2169498443 wait complete. 2024-04-25T15:02:27.214 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:02:27.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 -- 172.21.15.94:0/2976596911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd20108d90 msgr2=0x7fcd20115200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 --2- 172.21.15.94:0/2976596911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd20108d90 0x7fcd20115200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ca0 tx=0x7fcd1802a950 comp rx=0 tx=0).stop 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 -- 172.21.15.94:0/2976596911 shutdown_connections 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 --2- 172.21.15.94:0/2976596911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd20108d90 0x7fcd20115200 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 --2- 172.21.15.94:0/2976596911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd201083f0 0x7fcd20108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 --2- 172.21.15.94:0/2976596911 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2010e460 0x7fcd2010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 -- 172.21.15.94:0/2976596911 >> 172.21.15.94:0/2976596911 conn(0x7fcd20077ad0 msgr2=0x7fcd20079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 -- 172.21.15.94:0/2976596911 shutdown_connections 2024-04-25T15:02:27.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.412+0000 7fcd25600640 1 -- 172.21.15.94:0/2976596911 wait complete. 2024-04-25T15:02:27.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.413+0000 7fcd25600640 1 Processor -- start 2024-04-25T15:02:27.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.413+0000 7fcd25600640 1 -- start start 2024-04-25T15:02:27.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd25600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd201083f0 0x7fcd201a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd25600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20108d90 0x7fcd201a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd25600640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2010e460 0x7fcd2019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd25600640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201185d0 con 0x7fcd20108d90 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd25600640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20118450 con 0x7fcd2010e460 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd25600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20118750 con 0x7fcd201083f0 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20108d90 0x7fcd201a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd1fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd201083f0 0x7fcd201a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20108d90 0x7fcd201a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46322/0 (socket says 172.21.15.94:46322) 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd1f400640 1 -- 172.21.15.94:0/864192813 learned_addr learned my addr 172.21.15.94:0/864192813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:27.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.414+0000 7fcd24c00640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2010e460 0x7fcd2019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1fe00640 1 -- 172.21.15.94:0/864192813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2010e460 msgr2=0x7fcd2019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1fe00640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2010e460 0x7fcd2019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1fe00640 1 -- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20108d90 msgr2=0x7fcd201a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1fe00640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20108d90 0x7fcd201a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1fe00640 1 -- 172.21.15.94:0/864192813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2019eff0 con 0x7fcd201083f0 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1fe00640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd201083f0 0x7fcd201a46c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e9f0 tx=0x7fcd1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1cc00640 1 -- 172.21.15.94:0/864192813 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000ccc0 con 0x7fcd201083f0 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2019f2e0 con 0x7fcd201083f0 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1cc00640 1 -- 172.21.15.94:0/864192813 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd100108e0 con 0x7fcd201083f0 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd1cc00640 1 -- 172.21.15.94:0/864192813 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10018880 con 0x7fcd201083f0 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.415+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd201ab580 con 0x7fcd201083f0 2024-04-25T15:02:27.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.417+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd200690e0 con 0x7fcd201083f0 2024-04-25T15:02:27.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.418+0000 7fcd1cc00640 1 -- 172.21.15.94:0/864192813 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fcd10010430 con 0x7fcd201083f0 2024-04-25T15:02:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.420+0000 7fcd1cc00640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcd040a3db0 0x7fcd040a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.420+0000 7fcd1cc00640 1 -- 172.21.15.94:0/864192813 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7fcd100d0cf0 con 0x7fcd201083f0 2024-04-25T15:02:27.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.420+0000 7fcd1f400640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcd040a3db0 0x7fcd040a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.421+0000 7fcd1f400640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcd040a3db0 0x7fcd040a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd2010db00 tx=0x7fcd0c005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:27.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.424+0000 7fcd1cc00640 1 -- 172.21.15.94:0/864192813 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd1009cb80 con 0x7fcd201083f0 2024-04-25T15:02:27.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.649+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd2002d5c0 con 0x7fcd201083f0 2024-04-25T15:02:27.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.650+0000 7fcd1cc00640 1 -- 172.21.15.94:0/864192813 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7fcd100a0e80 con 0x7fcd201083f0 2024-04-25T15:02:27.653 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:27.653 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:02:27.653 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:02:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcd040a3db0 msgr2=0x7fcd040a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcd040a3db0 0x7fcd040a6270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcd2010db00 tx=0x7fcd0c005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd201083f0 msgr2=0x7fcd201a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd201083f0 0x7fcd201a46c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e9f0 tx=0x7fcd1000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 shutdown_connections 2024-04-25T15:02:27.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fcd040a3db0 0x7fcd040a6270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fcd2010e460 0x7fcd2019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd20108d90 0x7fcd201a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 --2- 172.21.15.94:0/864192813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd201083f0 0x7fcd201a46c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 >> 172.21.15.94:0/864192813 conn(0x7fcd20077ad0 msgr2=0x7fcd200791e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:27.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 shutdown_connections 2024-04-25T15:02:27.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.653+0000 7fcd25600640 1 -- 172.21.15.94:0/864192813 wait complete. 2024-04-25T15:02:27.670 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:27 smithi094 ceph-mon[27862]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:02:27.671 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:27 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1525370106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.671 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:27 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3083660579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.671 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:27 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2169498443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:02:27.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:27 smithi138 ceph-mon[32875]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:02:27.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:27 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1525370106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:27 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3083660579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:27 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2169498443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:27 smithi137 ceph-mon[32439]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 76 KiB/s wr, 43 op/s 2024-04-25T15:02:27.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:27 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1525370106' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:27 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3083660579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:27 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2169498443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 -- 172.21.15.94:0/4126288986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 msgr2=0x7febf0102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 --2- 172.21.15.94:0/4126288986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf0102400 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7febe4009900 tx=0x7febe402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 -- 172.21.15.94:0/4126288986 shutdown_connections 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 --2- 172.21.15.94:0/4126288986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febf01070f0 0x7febf010ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 --2- 172.21.15.94:0/4126288986 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf01029d0 0x7febf0106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 --2- 172.21.15.94:0/4126288986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf0102400 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 -- 172.21.15.94:0/4126288986 >> 172.21.15.94:0/4126288986 conn(0x7febf00fd8c0 msgr2=0x7febf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:27.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 -- 172.21.15.94:0/4126288986 shutdown_connections 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.861+0000 7febf6a00640 1 -- 172.21.15.94:0/4126288986 wait complete. 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.862+0000 7febf6a00640 1 Processor -- start 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.862+0000 7febf6a00640 1 -- start start 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf6a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf01986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf6a00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf01070f0 0x7febf0198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febf019cf40 0x7febf019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf6a00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7febf0112060 con 0x7febf0102020 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf6a00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7febf0111ee0 con 0x7febf01070f0 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf6a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febf01121e0 con 0x7febf019cf40 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf01986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf01986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46340/0 (socket says 172.21.15.94:46340) 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 -- 172.21.15.94:0/3011345707 learned_addr learned my addr 172.21.15.94:0/3011345707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf4c00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf01070f0 0x7febf0198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf6000640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febf019cf40 0x7febf019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 -- 172.21.15.94:0/3011345707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febf019cf40 msgr2=0x7febf019d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febf019cf40 0x7febf019d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 -- 172.21.15.94:0/3011345707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf01070f0 msgr2=0x7febf0198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf01070f0 0x7febf0198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.863+0000 7febf5600640 1 -- 172.21.15.94:0/3011345707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf019da30 con 0x7febf0102020 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.864+0000 7febf5600640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf01986d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febe4005940 tx=0x7febe40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:27.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.864+0000 7febee000640 1 -- 172.21.15.94:0/3011345707 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe403f030 con 0x7febf0102020 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.864+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf019dcc0 con 0x7febf0102020 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.864+0000 7febee000640 1 -- 172.21.15.94:0/3011345707 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febe40044c0 con 0x7febf0102020 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.864+0000 7febee000640 1 -- 172.21.15.94:0/3011345707 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4035070 con 0x7febf0102020 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.864+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf01a5480 con 0x7febf0102020 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.865+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf00690e0 con 0x7febf0102020 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.867+0000 7febee000640 1 -- 172.21.15.94:0/3011345707 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7febe4042030 con 0x7febf0102020 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.868+0000 7febee000640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7febd80a3db0 0x7febd80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:02:27.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.869+0000 7febee000640 1 -- 172.21.15.94:0/3011345707 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8498+0+0 (secure 0 0 0) 0x7febe40f0de0 con 0x7febf0102020 2024-04-25T15:02:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.869+0000 7febf4c00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7febd80a3db0 0x7febd80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:02:27.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.870+0000 7febf4c00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7febd80a3db0 0x7febd80a6270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7febdc0095f0 tx=0x7febdc005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:02:27.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:27.874+0000 7febee000640 1 -- 172.21.15.94:0/3011345707 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febe40bd2c0 con 0x7febf0102020 2024-04-25T15:02:28.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.078+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febf01029d0 con 0x7febf0102020 2024-04-25T15:02:28.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.082+0000 7febee000640 1 -- 172.21.15.94:0/3011345707 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23061 (secure 0 0 0) 0x7febe40bcbc0 con 0x7febf0102020 2024-04-25T15:02:28.085 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:02:28.086 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":89,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:02:09.415151+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/702744035":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3354212797":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6801/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1994260887":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/2523030980":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6800/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:02:28.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7febd80a3db0 msgr2=0x7febd80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:28.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7febd80a3db0 0x7febd80a6270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7febdc0095f0 tx=0x7febdc005f90 comp rx=0 tx=0).stop 2024-04-25T15:02:28.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 msgr2=0x7febf01986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:02:28.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf01986d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7febe4005940 tx=0x7febe40040c0 comp rx=0 tx=0).stop 2024-04-25T15:02:28.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 shutdown_connections 2024-04-25T15:02:28.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7febd80a3db0 0x7febd80a6270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:28.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febf019cf40 0x7febf019d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:28.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7febf01070f0 0x7febf0198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:28.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 --2- 172.21.15.94:0/3011345707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febf0102020 0x7febf01986d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:02:28.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 >> 172.21.15.94:0/3011345707 conn(0x7febf00fd8c0 msgr2=0x7febf00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:02:28.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 shutdown_connections 2024-04-25T15:02:28.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:02:28.085+0000 7febf6a00640 1 -- 172.21.15.94:0/3011345707 wait complete. 2024-04-25T15:02:28.107 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir 2024-04-25T15:02:28.291 INFO:teuthology.orchestra.run.smithi094.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2024-04-25T15:02:28.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T15:02:28.299 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 403, in task quiescer = Quiescer(ctx=ctx, fscid=fs['id'], mds_cluster=mds_cluster, cluster_name=cluster_name, **config) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 72, in __init__ self.run_dir = mds_cluster.get_config("run_dir") File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 353, in get_config return super(MDSClusterBase, self).get_config(key, service_type) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 249, in get_config return self.json_asok(['config', 'get', key], service_type, service_id)[key] File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 266, in json_asok proc = self.mon_manager.admin_socket(service_type, service_id, command, timeout=timeout) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/ceph_manager.py", line 1922, in admin_socket return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir' 2024-04-25T15:02:28.548 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b8a21f783bb848eb93ca47e55fbcaf13 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 403, in task quiescer = Quiescer(ctx=ctx, fscid=fs['id'], mds_cluster=mds_cluster, cluster_name=cluster_name, **config) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/quiescer.py", line 72, in __init__ self.run_dir = mds_cluster.get_config("run_dir") File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 353, in get_config return super(MDSClusterBase, self).get_config(key, service_type) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 249, in get_config return self.json_asok(['config', 'get', key], service_type, service_id)[key] File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/cephfs/filesystem.py", line 266, in json_asok proc = self.mon_manager.admin_socket(service_type, service_id, command, timeout=timeout) File "/home/teuthworker/src/github.com_ceph_ceph_107813aeb7aa17f2c7d5c13d9b72e8f5fe87fd69/qa/tasks/ceph_manager.py", line 1922, in admin_socket return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi094 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir' 2024-04-25T15:02:28.552 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T15:02:28.738 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T15:02:28.764 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:02:28.768 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:28 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/864192813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:28.769 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:28 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3011345707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:28.769 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:28 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/864192813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:28.769 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:28 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3011345707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:28.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:28 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/864192813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:02:28.769 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:28 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3011345707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:02:28.791 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T15:02:28.801 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T15:02:28.801 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T15:02:28.801 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:02:28.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T15:02:28.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T15:02:29.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:29 smithi138 ceph-mon[32875]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-25T15:02:29.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:29 smithi094 ceph-mon[27862]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-25T15:02:29.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:29 smithi137 ceph-mon[32439]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-04-25T15:02:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:31 smithi138 ceph-mon[32875]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T15:02:31.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:31 smithi094 ceph-mon[27862]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T15:02:31.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:31 smithi137 ceph-mon[32439]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 597 B/s wr, 2 op/s 2024-04-25T15:02:33.534 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:33 smithi137 ceph-mon[32439]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:02:33.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:33 smithi138 ceph-mon[32875]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:02:33.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:33 smithi094 ceph-mon[27862]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:02:34.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:34 smithi138 ceph-mon[32875]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:02:34.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:34 smithi094 ceph-mon[27862]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:02:34.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:34 smithi137 ceph-mon[32439]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:02:36.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:36 smithi138 ceph-mon[32875]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:02:36.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:36 smithi094 ceph-mon[27862]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:02:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:36 smithi137 ceph-mon[32439]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T15:02:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:38 smithi138 ceph-mon[32875]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:02:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:38 smithi094 ceph-mon[27862]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:02:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:38 smithi137 ceph-mon[32439]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T15:02:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:40 smithi138 ceph-mon[32875]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 4 op/s 2024-04-25T15:02:40.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:40 smithi094 ceph-mon[27862]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 4 op/s 2024-04-25T15:02:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:40 smithi137 ceph-mon[32439]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 4 op/s 2024-04-25T15:02:42.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:42 smithi094 ceph-mon[27862]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T15:02:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:42 smithi138 ceph-mon[32875]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T15:02:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:42 smithi137 ceph-mon[32439]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T15:02:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:44 smithi138 ceph-mon[32875]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:44 smithi094 ceph-mon[27862]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:44.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:44 smithi137 ceph-mon[32439]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:46 smithi138 ceph-mon[32875]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:46.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:46 smithi094 ceph-mon[27862]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:46 smithi137 ceph-mon[32439]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:49.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:49 smithi138 ceph-mon[32875]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:49.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:49 smithi094 ceph-mon[27862]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:49.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:49 smithi137 ceph-mon[32439]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:51.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:51 smithi138 ceph-mon[32875]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:51.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:51 smithi094 ceph-mon[27862]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:51.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:51 smithi137 ceph-mon[32439]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:53.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:53 smithi094 ceph-mon[27862]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:53.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:53 smithi138 ceph-mon[32875]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:53.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:53 smithi137 ceph-mon[32439]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:55.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:55 smithi138 ceph-mon[32875]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:55.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:55 smithi094 ceph-mon[27862]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:55.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:55 smithi137 ceph-mon[32439]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:57 smithi138 ceph-mon[32875]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:57.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:57 smithi094 ceph-mon[27862]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:57 smithi137 ceph-mon[32439]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:02:59.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:02:59 smithi138 ceph-mon[32875]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:59.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:02:59 smithi094 ceph-mon[27862]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:02:59.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:02:59 smithi137 ceph-mon[32439]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:00.463 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:03:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:03:00.124+0000 7ff5d1800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:03:00.463 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:03:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:03:00.124+0000 7ff5d1800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:03:00.463 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:03:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:03:00.124+0000 7ff5d1800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:03:00.463 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:03:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:03:00.124+0000 7ff5d1800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:03:00.463 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:03:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:03:00.124+0000 7ff5d1800700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T15:03:01.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:01 smithi138 ceph-mon[32875]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:01.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:01 smithi094 ceph-mon[27862]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:01.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:01 smithi137 ceph-mon[32439]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:03.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:03 smithi138 ceph-mon[32875]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:03.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:03 smithi138 ceph-mon[32875]: mgrmap e25: x(active, since 52s), standbys: z, y 2024-04-25T15:03:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:03 smithi094 ceph-mon[27862]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:03 smithi094 ceph-mon[27862]: mgrmap e25: x(active, since 52s), standbys: z, y 2024-04-25T15:03:03.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:03 smithi137 ceph-mon[32439]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:03.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:03 smithi137 ceph-mon[32439]: mgrmap e25: x(active, since 52s), standbys: z, y 2024-04-25T15:03:05.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:05 smithi138 ceph-mon[32875]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:05.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:05 smithi094 ceph-mon[27862]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:05.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:05 smithi137 ceph-mon[32439]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:07 smithi138 ceph-mon[32875]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:07.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:07 smithi094 ceph-mon[27862]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:07.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:07 smithi137 ceph-mon[32439]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:09.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:09 smithi138 ceph-mon[32875]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:09.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:09 smithi094 ceph-mon[27862]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:09.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:09 smithi137 ceph-mon[32439]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:11.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:11 smithi138 ceph-mon[32875]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:11.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:11 smithi094 ceph-mon[27862]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:11.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:11 smithi137 ceph-mon[32439]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:03:13.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:13 smithi138 ceph-mon[32875]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:13.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:13 smithi094 ceph-mon[27862]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:13.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:13 smithi137 ceph-mon[32439]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:03:15.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:15 smithi138 ceph-mon[32875]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-04-25T15:03:15.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:15 smithi094 ceph-mon[27862]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-04-25T15:03:15.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:15 smithi137 ceph-mon[32439]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 170 B/s wr, 1 op/s 2024-04-25T15:03:17.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:17 smithi138 ceph-mon[32875]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:03:17.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:17 smithi094 ceph-mon[27862]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:03:17.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:17 smithi137 ceph-mon[32439]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:03:19.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:19 smithi138 ceph-mon[32875]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:03:19.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:19 smithi094 ceph-mon[27862]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:03:19.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:19 smithi137 ceph-mon[32439]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:03:20.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:20 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:03:20.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:20 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:03:20.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:20 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:03:21.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:21 smithi138 ceph-mon[32875]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:03:21.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:21 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:21 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:21 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:21 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:21 smithi094 ceph-mon[27862]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:03:21.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:21 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:21 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:21 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:21 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:21 smithi137 ceph-mon[32439]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:03:21.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:21 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:21 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:21 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:21.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:21 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:22 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:03:22.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:22 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:03:22.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:22 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:22.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:22 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:03:22.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:22 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:03:22.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:22 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:22 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:03:22.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:22 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:03:22.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:22 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:03:23.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:23 smithi138 ceph-mon[32875]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:03:23.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:23 smithi094 ceph-mon[27862]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:03:23.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:23 smithi137 ceph-mon[32439]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:03:25.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:25 smithi138 ceph-mon[32875]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:03:25.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:25 smithi094 ceph-mon[27862]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:03:25.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:25 smithi137 ceph-mon[32439]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:03:27.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:27 smithi138 ceph-mon[32875]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:03:27.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:27 smithi094 ceph-mon[27862]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:03:27.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:27 smithi137 ceph-mon[32439]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:03:29.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:29 smithi138 ceph-mon[32875]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:29.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:29 smithi094 ceph-mon[27862]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:29.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:29 smithi137 ceph-mon[32439]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:31.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:31 smithi138 ceph-mon[32875]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:31.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:31 smithi094 ceph-mon[27862]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:31.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:31 smithi137 ceph-mon[32439]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:33.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:33 smithi138 ceph-mon[32875]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:33.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:33 smithi094 ceph-mon[27862]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:33.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:33 smithi137 ceph-mon[32439]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:35.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:35 smithi138 ceph-mon[32875]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:35.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:35 smithi094 ceph-mon[27862]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:35.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:35 smithi137 ceph-mon[32439]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:37.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:37 smithi138 ceph-mon[32875]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:37.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:37 smithi094 ceph-mon[27862]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:37.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:37 smithi137 ceph-mon[32439]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:03:39.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:39 smithi138 ceph-mon[32875]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:39.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:39 smithi094 ceph-mon[27862]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:39.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:39 smithi137 ceph-mon[32439]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:41.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:41 smithi138 ceph-mon[32875]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:03:41.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:41 smithi094 ceph-mon[27862]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:03:41.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:41 smithi137 ceph-mon[32439]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:03:43.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:43 smithi138 ceph-mon[32875]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:43.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:43 smithi138 ceph-mon[32875]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T15:03:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:43 smithi094 ceph-mon[27862]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:43 smithi094 ceph-mon[27862]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T15:03:43.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:43 smithi137 ceph-mon[32439]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:03:43.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:43 smithi137 ceph-mon[32439]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-25T15:03:45.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:45 smithi138 ceph-mon[32875]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:03:45.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:45 smithi094 ceph-mon[27862]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:03:45.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:45 smithi137 ceph-mon[32439]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-25T15:03:47.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:47 smithi138 ceph-mon[32875]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:47.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:47 smithi094 ceph-mon[27862]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:47.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:47 smithi137 ceph-mon[32439]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:49.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:49 smithi094 ceph-mon[27862]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:49.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:49 smithi137 ceph-mon[32439]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:49.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:49 smithi138 ceph-mon[32875]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:51.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:51 smithi137 ceph-mon[32439]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:51.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:51 smithi138 ceph-mon[32875]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:51.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:51 smithi094 ceph-mon[27862]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:53.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:53 smithi137 ceph-mon[32439]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:53.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:53 smithi138 ceph-mon[32875]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:53 smithi094 ceph-mon[27862]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-25T15:03:55.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:55 smithi137 ceph-mon[32439]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:03:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:55 smithi138 ceph-mon[32875]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:03:55.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:55 smithi094 ceph-mon[27862]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:03:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:57 smithi137 ceph-mon[32439]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:03:57.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:57 smithi138 ceph-mon[32875]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:03:57.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:57 smithi094 ceph-mon[27862]: pgmap v56: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:03:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:03:59 smithi138 ceph-mon[32875]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:59.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:03:59 smithi094 ceph-mon[27862]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:03:59.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:03:59 smithi137 ceph-mon[32439]: pgmap v57: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:01 smithi138 ceph-mon[32875]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:01 smithi094 ceph-mon[27862]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:01.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:01 smithi137 ceph-mon[32439]: pgmap v58: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:03.534 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:03 smithi137 ceph-mon[32439]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:03.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:03 smithi138 ceph-mon[32875]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:03.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:03 smithi094 ceph-mon[27862]: pgmap v59: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:05.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:05 smithi138 ceph-mon[32875]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:05.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:05 smithi094 ceph-mon[27862]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:05.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:05 smithi137 ceph-mon[32439]: pgmap v60: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:07.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:07 smithi138 ceph-mon[32875]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:07 smithi094 ceph-mon[27862]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:07 smithi137 ceph-mon[32439]: pgmap v61: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:09 smithi138 ceph-mon[32875]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:09.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:09 smithi094 ceph-mon[27862]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:09 smithi137 ceph-mon[32439]: pgmap v62: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:11.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:11 smithi094 ceph-mon[27862]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:11.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:11 smithi138 ceph-mon[32875]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:11 smithi137 ceph-mon[32439]: pgmap v63: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:04:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:13 smithi138 ceph-mon[32875]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:13.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:13 smithi094 ceph-mon[27862]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:13 smithi137 ceph-mon[32439]: pgmap v64: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:04:15.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:15 smithi138 ceph-mon[32875]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:04:15.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:15 smithi094 ceph-mon[27862]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:04:15.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:15 smithi137 ceph-mon[32439]: pgmap v65: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:04:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:17 smithi138 ceph-mon[32875]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:04:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:17 smithi094 ceph-mon[27862]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:04:17.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:17 smithi137 ceph-mon[32439]: pgmap v66: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-25T15:04:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:19 smithi138 ceph-mon[32875]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:04:19.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:19 smithi094 ceph-mon[27862]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:04:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:19 smithi137 ceph-mon[32439]: pgmap v67: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:04:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:21 smithi138 ceph-mon[32875]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:04:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:21 smithi094 ceph-mon[27862]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:04:21.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:21 smithi137 ceph-mon[32439]: pgmap v68: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:04:22.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:22 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:22.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:22 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:22.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:22 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:04:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:23 smithi138 ceph-mon[32875]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:04:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:23 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:23 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:23 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:04:23.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:23 smithi094 ceph-mon[27862]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:04:23.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:23 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:23.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:23 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:23.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:23 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:04:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:23 smithi137 ceph-mon[32439]: pgmap v69: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:04:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:23 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:04:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:23 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:04:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:23 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:04:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:24 smithi138 ceph-mon[32875]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:24 smithi094 ceph-mon[27862]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:24 smithi137 ceph-mon[32439]: pgmap v70: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:26.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:26 smithi138 ceph-mon[32875]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:26.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:26 smithi094 ceph-mon[27862]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:26 smithi137 ceph-mon[32439]: pgmap v71: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:28.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:28 smithi138 ceph-mon[32875]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:28 smithi094 ceph-mon[27862]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:28 smithi137 ceph-mon[32439]: pgmap v72: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:30.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:30 smithi138 ceph-mon[32875]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:30.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:30 smithi094 ceph-mon[27862]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:30 smithi137 ceph-mon[32439]: pgmap v73: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:32.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:32 smithi138 ceph-mon[32875]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:32.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:32 smithi094 ceph-mon[27862]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:32.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:32 smithi137 ceph-mon[32439]: pgmap v74: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:34.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:34 smithi138 ceph-mon[32875]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:34.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:34 smithi094 ceph-mon[27862]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:34.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:34 smithi137 ceph-mon[32439]: pgmap v75: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:36.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:36 smithi138 ceph-mon[32875]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:36.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:36 smithi094 ceph-mon[27862]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:36 smithi137 ceph-mon[32439]: pgmap v76: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:38 smithi138 ceph-mon[32875]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:38 smithi094 ceph-mon[27862]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:38.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:38 smithi137 ceph-mon[32439]: pgmap v77: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:40 smithi138 ceph-mon[32875]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:40.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:40 smithi094 ceph-mon[27862]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:40.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:40 smithi137 ceph-mon[32439]: pgmap v78: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:04:42.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:42 smithi138 ceph-mon[32875]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:42.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:42 smithi094 ceph-mon[27862]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:42.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:42 smithi137 ceph-mon[32439]: pgmap v79: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:44.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:44 smithi138 ceph-mon[32875]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:44.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:44 smithi094 ceph-mon[27862]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:44.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:44 smithi137 ceph-mon[32439]: pgmap v80: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:46.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:46 smithi138 ceph-mon[32875]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:46.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:46 smithi094 ceph-mon[27862]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:46.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:46 smithi137 ceph-mon[32439]: pgmap v81: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:48.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:48 smithi138 ceph-mon[32875]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:48.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:48 smithi094 ceph-mon[27862]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:48.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:48 smithi137 ceph-mon[32439]: pgmap v82: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:50.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:50 smithi138 ceph-mon[32875]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:50.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:50 smithi094 ceph-mon[27862]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:50.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:50 smithi137 ceph-mon[32439]: pgmap v83: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:52.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:52 smithi138 ceph-mon[32875]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:52.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:52 smithi094 ceph-mon[27862]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:52.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:52 smithi137 ceph-mon[32439]: pgmap v84: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:54.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:54 smithi138 ceph-mon[32875]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:54.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:54 smithi094 ceph-mon[27862]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:54.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:54 smithi137 ceph-mon[32439]: pgmap v85: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:56.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:56 smithi138 ceph-mon[32875]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:56.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:56 smithi094 ceph-mon[27862]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:56.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:56 smithi137 ceph-mon[32439]: pgmap v86: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:04:58.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:04:58 smithi138 ceph-mon[32875]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:58.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:04:58 smithi094 ceph-mon[27862]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:04:58.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:04:58 smithi137 ceph-mon[32439]: pgmap v87: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:00.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:00 smithi138 ceph-mon[32875]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:00.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:00 smithi094 ceph-mon[27862]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:00.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:00 smithi137 ceph-mon[32439]: pgmap v88: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:02.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:02 smithi138 ceph-mon[32875]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T15:05:02.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:02 smithi094 ceph-mon[27862]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T15:05:02.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:02 smithi137 ceph-mon[32439]: pgmap v89: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 11 KiB/s wr, 5 op/s 2024-04-25T15:05:04.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:04 smithi138 ceph-mon[32875]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T15:05:04.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:04 smithi094 ceph-mon[27862]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T15:05:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:04 smithi137 ceph-mon[32439]: pgmap v90: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-25T15:05:06.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:06 smithi138 ceph-mon[32875]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:06.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:06 smithi094 ceph-mon[27862]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:06.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:06 smithi137 ceph-mon[32439]: pgmap v91: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:08.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:08 smithi138 ceph-mon[32875]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:08.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:08 smithi094 ceph-mon[27862]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:08.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:08 smithi137 ceph-mon[32439]: pgmap v92: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:10.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:10 smithi138 ceph-mon[32875]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:10.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:10 smithi094 ceph-mon[27862]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:10.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:10 smithi137 ceph-mon[32439]: pgmap v93: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:05:12.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:12 smithi138 ceph-mon[32875]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:12.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:12 smithi094 ceph-mon[27862]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:12.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:12 smithi137 ceph-mon[32439]: pgmap v94: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:05:14.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:14 smithi138 ceph-mon[32875]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T15:05:14.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:14 smithi094 ceph-mon[27862]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T15:05:14.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:14 smithi137 ceph-mon[32439]: pgmap v95: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 11 KiB/s wr, 14 op/s 2024-04-25T15:05:16.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:16 smithi138 ceph-mon[32875]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T15:05:16.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:16 smithi094 ceph-mon[27862]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T15:05:16.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:16 smithi137 ceph-mon[32439]: pgmap v96: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-25T15:05:18.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:18 smithi138 ceph-mon[32875]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:18.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:18 smithi094 ceph-mon[27862]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:18.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:18 smithi137 ceph-mon[32439]: pgmap v97: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:20.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:20 smithi138 ceph-mon[32875]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:20.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:20 smithi094 ceph-mon[27862]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:20.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:20 smithi137 ceph-mon[32439]: pgmap v98: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:05:22.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:22 smithi138 ceph-mon[32875]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:05:22.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:22 smithi094 ceph-mon[27862]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:05:22.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:22 smithi137 ceph-mon[32439]: pgmap v99: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:05:23.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:23 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:05:24.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:23 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:05:24.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:23 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:05:24.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:24 smithi137 ceph-mon[32439]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:24.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:24 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:05:24.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:24 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:05:24.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:24 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:05:25.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:24 smithi138 ceph-mon[32875]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:25.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:24 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:05:25.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:24 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:05:25.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:24 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:05:25.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:24 smithi094 ceph-mon[27862]: pgmap v100: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:25.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:24 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:05:25.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:24 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:05:25.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:24 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:05:26.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:26 smithi137 ceph-mon[32439]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:27.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:26 smithi138 ceph-mon[32875]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:27.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:26 smithi094 ceph-mon[27862]: pgmap v101: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:28 smithi137 ceph-mon[32439]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:29.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:28 smithi138 ceph-mon[32875]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:29.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:28 smithi094 ceph-mon[27862]: pgmap v102: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:30 smithi138 ceph-mon[32875]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:31.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:30 smithi094 ceph-mon[27862]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:31.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:30 smithi137 ceph-mon[32439]: pgmap v103: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:33.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:32 smithi138 ceph-mon[32875]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:33.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:32 smithi094 ceph-mon[27862]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:33.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:32 smithi137 ceph-mon[32439]: pgmap v104: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:35.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:34 smithi137 ceph-mon[32439]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:35.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:34 smithi138 ceph-mon[32875]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:35.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:34 smithi094 ceph-mon[27862]: pgmap v105: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:37.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:36 smithi138 ceph-mon[32875]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:37.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:36 smithi094 ceph-mon[27862]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:37.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:36 smithi137 ceph-mon[32439]: pgmap v106: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:39.010 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:38 smithi138 ceph-mon[32875]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:39.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:38 smithi094 ceph-mon[27862]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:39.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:38 smithi137 ceph-mon[32439]: pgmap v107: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:41.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:40 smithi138 ceph-mon[32875]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:41.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:40 smithi094 ceph-mon[27862]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:41.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:40 smithi137 ceph-mon[32439]: pgmap v108: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:43.015 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:42 smithi094 ceph-mon[27862]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:43.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:42 smithi138 ceph-mon[32875]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:43.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:42 smithi137 ceph-mon[32439]: pgmap v109: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:45.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:44 smithi138 ceph-mon[32875]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:45.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:44 smithi094 ceph-mon[27862]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:45.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:44 smithi137 ceph-mon[32439]: pgmap v110: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:47.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:46 smithi138 ceph-mon[32875]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:47.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:46 smithi094 ceph-mon[27862]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:47.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:46 smithi137 ceph-mon[32439]: pgmap v111: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:49.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:48 smithi138 ceph-mon[32875]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:49.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:48 smithi094 ceph-mon[27862]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:49.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:48 smithi137 ceph-mon[32439]: pgmap v112: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:51.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:50 smithi138 ceph-mon[32875]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:51.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:50 smithi094 ceph-mon[27862]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:51.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:50 smithi137 ceph-mon[32439]: pgmap v113: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:53.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:52 smithi138 ceph-mon[32875]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:53.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:52 smithi094 ceph-mon[27862]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:53.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:52 smithi137 ceph-mon[32439]: pgmap v114: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:55.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:54 smithi138 ceph-mon[32875]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:55.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:54 smithi094 ceph-mon[27862]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:55.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:54 smithi137 ceph-mon[32439]: pgmap v115: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:57.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:56 smithi138 ceph-mon[32875]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:57.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:56 smithi094 ceph-mon[27862]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:57.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:56 smithi137 ceph-mon[32439]: pgmap v116: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:05:59.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:05:58 smithi138 ceph-mon[32875]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:59.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:05:58 smithi094 ceph-mon[27862]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:05:59.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:05:58 smithi137 ceph-mon[32439]: pgmap v117: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:01.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:00 smithi138 ceph-mon[32875]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:01.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:00 smithi094 ceph-mon[27862]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:01.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:00 smithi137 ceph-mon[32439]: pgmap v118: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:03.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:02 smithi138 ceph-mon[32875]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:03.254 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:02 smithi137 ceph-mon[32439]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:03.254 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:02 smithi094 ceph-mon[27862]: pgmap v119: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:05.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:04 smithi138 ceph-mon[32875]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:05.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:04 smithi094 ceph-mon[27862]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:05.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:04 smithi137 ceph-mon[32439]: pgmap v120: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:07.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:06 smithi138 ceph-mon[32875]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:07.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:06 smithi094 ceph-mon[27862]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:07.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:06 smithi137 ceph-mon[32439]: pgmap v121: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:09.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:08 smithi138 ceph-mon[32875]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:09.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:08 smithi094 ceph-mon[27862]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:09.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:08 smithi137 ceph-mon[32439]: pgmap v122: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:11.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:10 smithi138 ceph-mon[32875]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:11.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:10 smithi094 ceph-mon[27862]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:11.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:10 smithi137 ceph-mon[32439]: pgmap v123: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:12 smithi138 ceph-mon[32875]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:13.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:12 smithi094 ceph-mon[27862]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:13.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:12 smithi137 ceph-mon[32439]: pgmap v124: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:14 smithi138 ceph-mon[32875]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:15.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:14 smithi094 ceph-mon[27862]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:15.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:14 smithi137 ceph-mon[32439]: pgmap v125: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:06:17.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:16 smithi138 ceph-mon[32875]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:17.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:16 smithi094 ceph-mon[27862]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:17.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:16 smithi137 ceph-mon[32439]: pgmap v126: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:06:19.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:18 smithi138 ceph-mon[32875]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:19.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:18 smithi094 ceph-mon[27862]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:19.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:18 smithi137 ceph-mon[32439]: pgmap v127: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:21.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:20 smithi138 ceph-mon[32875]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:21.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:20 smithi094 ceph-mon[27862]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:21.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:20 smithi137 ceph-mon[32439]: pgmap v128: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:06:23.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:22 smithi138 ceph-mon[32875]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:06:23.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:22 smithi094 ceph-mon[27862]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:06:23.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:22 smithi137 ceph-mon[32439]: pgmap v129: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:06:25.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:24 smithi138 ceph-mon[32875]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:06:25.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:24 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:06:25.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:24 smithi094 ceph-mon[27862]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:06:25.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:24 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:06:25.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:24 smithi137 ceph-mon[32439]: pgmap v130: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:06:25.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:24 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:06:27.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:26 smithi138 ceph-mon[32875]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:06:27.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:26 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:06:27.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:26 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:06:27.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:26 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:06:27.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:26 smithi094 ceph-mon[27862]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:06:27.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:26 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:06:27.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:26 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:06:27.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:26 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:06:27.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:26 smithi137 ceph-mon[32439]: pgmap v131: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:06:27.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:26 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:06:27.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:26 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:06:27.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:26 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:06:29.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:28 smithi138 ceph-mon[32875]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:29.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:28 smithi094 ceph-mon[27862]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:29.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:28 smithi137 ceph-mon[32439]: pgmap v132: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:30 smithi138 ceph-mon[32875]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:31.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:30 smithi094 ceph-mon[27862]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:31.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:30 smithi137 ceph-mon[32439]: pgmap v133: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:33.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:32 smithi138 ceph-mon[32875]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:33.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:32 smithi094 ceph-mon[27862]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:33.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:32 smithi137 ceph-mon[32439]: pgmap v134: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:35.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:34 smithi138 ceph-mon[32875]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:35.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:34 smithi094 ceph-mon[27862]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:35.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:34 smithi137 ceph-mon[32439]: pgmap v135: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:37.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:36 smithi138 ceph-mon[32875]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:37.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:36 smithi094 ceph-mon[27862]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:37.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:36 smithi137 ceph-mon[32439]: pgmap v136: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:39.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:38 smithi138 ceph-mon[32875]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:39.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:38 smithi094 ceph-mon[27862]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:39.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:38 smithi137 ceph-mon[32439]: pgmap v137: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:41.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:40 smithi138 ceph-mon[32875]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:41.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:40 smithi094 ceph-mon[27862]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:41.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:40 smithi137 ceph-mon[32439]: pgmap v138: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:43.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:42 smithi138 ceph-mon[32875]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:43.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:42 smithi094 ceph-mon[27862]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:43.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:42 smithi137 ceph-mon[32439]: pgmap v139: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:45.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:44 smithi138 ceph-mon[32875]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:45.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:44 smithi094 ceph-mon[27862]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:45.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:44 smithi137 ceph-mon[32439]: pgmap v140: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:47.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:46 smithi138 ceph-mon[32875]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:47.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:46 smithi094 ceph-mon[27862]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:47.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:46 smithi137 ceph-mon[32439]: pgmap v141: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:49.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:49 smithi138 ceph-mon[32875]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:49.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:49 smithi094 ceph-mon[27862]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:49.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:49 smithi137 ceph-mon[32439]: pgmap v142: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:51.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:51 smithi138 ceph-mon[32875]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:51.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:51 smithi094 ceph-mon[27862]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:51.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:51 smithi137 ceph-mon[32439]: pgmap v143: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:53.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:53 smithi138 ceph-mon[32875]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:53.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:53 smithi094 ceph-mon[27862]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:53.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:53 smithi137 ceph-mon[32439]: pgmap v144: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:55.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:55 smithi138 ceph-mon[32875]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:55.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:55 smithi094 ceph-mon[27862]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:55.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:55 smithi137 ceph-mon[32439]: pgmap v145: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:57 smithi138 ceph-mon[32875]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:57.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:57 smithi094 ceph-mon[27862]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:57 smithi137 ceph-mon[32439]: pgmap v146: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:06:59.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:06:59 smithi138 ceph-mon[32875]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:59.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:06:59 smithi094 ceph-mon[27862]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:06:59.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:06:59 smithi137 ceph-mon[32439]: pgmap v147: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:01.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:01 smithi138 ceph-mon[32875]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:01.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:01 smithi094 ceph-mon[27862]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:01.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:01 smithi137 ceph-mon[32439]: pgmap v148: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:03.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:03 smithi138 ceph-mon[32875]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:03 smithi094 ceph-mon[27862]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:03.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:03 smithi137 ceph-mon[32439]: pgmap v149: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:05.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:05 smithi138 ceph-mon[32875]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:05.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:05 smithi094 ceph-mon[27862]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:05.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:05 smithi137 ceph-mon[32439]: pgmap v150: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:07 smithi138 ceph-mon[32875]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:07.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:07 smithi094 ceph-mon[27862]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:07.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:07 smithi137 ceph-mon[32439]: pgmap v151: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:09.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:09 smithi138 ceph-mon[32875]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:09.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:09 smithi094 ceph-mon[27862]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:09.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:09 smithi137 ceph-mon[32439]: pgmap v152: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:11.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:11 smithi138 ceph-mon[32875]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:11.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:11 smithi094 ceph-mon[27862]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:11.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:11 smithi137 ceph-mon[32439]: pgmap v153: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:13.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:13 smithi138 ceph-mon[32875]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:13.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:13 smithi094 ceph-mon[27862]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:13.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:13 smithi137 ceph-mon[32439]: pgmap v154: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:15.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:15 smithi138 ceph-mon[32875]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:15.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:15 smithi094 ceph-mon[27862]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:15.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:15 smithi137 ceph-mon[32439]: pgmap v155: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:07:17.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:17 smithi138 ceph-mon[32875]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:17.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:17 smithi094 ceph-mon[27862]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:17.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:17 smithi137 ceph-mon[32439]: pgmap v156: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:07:19.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:19 smithi138 ceph-mon[32875]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:19.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:19 smithi094 ceph-mon[27862]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:19.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:19 smithi137 ceph-mon[32439]: pgmap v157: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:21.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:21 smithi138 ceph-mon[32875]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:21.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:21 smithi094 ceph-mon[27862]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:21.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:21 smithi137 ceph-mon[32439]: pgmap v158: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:07:23.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:23 smithi138 ceph-mon[32875]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:07:23.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:23 smithi094 ceph-mon[27862]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:07:23.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:23 smithi137 ceph-mon[32439]: pgmap v159: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:07:25.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:25 smithi138 ceph-mon[32875]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:07:25.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:25 smithi094 ceph-mon[27862]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:07:25.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:25 smithi137 ceph-mon[32439]: pgmap v160: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:07:27.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:27 smithi138 ceph-mon[32875]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:07:27.458 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:27 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:07:27.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:27 smithi094 ceph-mon[27862]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:07:27.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:27 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:07:27.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:27 smithi137 ceph-mon[32439]: pgmap v161: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:07:27.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:27 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:07:28.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:28 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:07:28.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:28 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:07:28.463 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:28 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:07:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:28 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:07:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:28 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:07:28.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:28 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:07:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:28 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:07:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:28 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:07:28.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:28 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:07:29.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:29 smithi094 ceph-mon[27862]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:29.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:29 smithi137 ceph-mon[32439]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:29.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:29 smithi138 ceph-mon[32875]: pgmap v162: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:31.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:31 smithi137 ceph-mon[32439]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:31 smithi138 ceph-mon[32875]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:31.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:31 smithi094 ceph-mon[27862]: pgmap v163: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:33.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:33 smithi137 ceph-mon[32439]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:33.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:33 smithi138 ceph-mon[32875]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:33.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:33 smithi094 ceph-mon[27862]: pgmap v164: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:35.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:35 smithi137 ceph-mon[32439]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:35.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:35 smithi138 ceph-mon[32875]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:35 smithi094 ceph-mon[27862]: pgmap v165: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:37.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:37 smithi137 ceph-mon[32439]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:37.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:37 smithi138 ceph-mon[32875]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:37.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:37 smithi094 ceph-mon[27862]: pgmap v166: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:39.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:39 smithi138 ceph-mon[32875]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:39.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:39 smithi094 ceph-mon[27862]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:39.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:39 smithi137 ceph-mon[32439]: pgmap v167: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:41 smithi138 ceph-mon[32875]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:41.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:41 smithi094 ceph-mon[27862]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:41.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:41 smithi137 ceph-mon[32439]: pgmap v168: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:07:43.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:43 smithi094 ceph-mon[27862]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:43 smithi138 ceph-mon[32875]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:43 smithi137 ceph-mon[32439]: pgmap v169: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:45.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:45 smithi138 ceph-mon[32875]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:45 smithi094 ceph-mon[27862]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:45.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:45 smithi137 ceph-mon[32439]: pgmap v170: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:47.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:47 smithi138 ceph-mon[32875]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:47.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:47 smithi094 ceph-mon[27862]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:47.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:47 smithi137 ceph-mon[32439]: pgmap v171: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:49.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:49 smithi138 ceph-mon[32875]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:49.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:49 smithi094 ceph-mon[27862]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:49.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:49 smithi137 ceph-mon[32439]: pgmap v172: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:51.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:51 smithi138 ceph-mon[32875]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:51.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:51 smithi094 ceph-mon[27862]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:51.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:51 smithi137 ceph-mon[32439]: pgmap v173: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:53.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:53 smithi138 ceph-mon[32875]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:53 smithi094 ceph-mon[27862]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:53.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:53 smithi137 ceph-mon[32439]: pgmap v174: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:55 smithi138 ceph-mon[32875]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:55.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:55 smithi094 ceph-mon[27862]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:55.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:55 smithi137 ceph-mon[32439]: pgmap v175: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:57.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:57 smithi138 ceph-mon[32875]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:57.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:57 smithi094 ceph-mon[27862]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:57.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:57 smithi137 ceph-mon[32439]: pgmap v176: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:07:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:07:59 smithi138 ceph-mon[32875]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:59.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:07:59 smithi094 ceph-mon[27862]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:07:59.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:07:59 smithi137 ceph-mon[32439]: pgmap v177: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:01 smithi138 ceph-mon[32875]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:01 smithi094 ceph-mon[27862]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:01.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:01 smithi137 ceph-mon[32439]: pgmap v178: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:03.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:03 smithi138 ceph-mon[32875]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:03.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:03 smithi094 ceph-mon[27862]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:03.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:03 smithi137 ceph-mon[32439]: pgmap v179: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:05.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:05 smithi138 ceph-mon[32875]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:05.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:05 smithi094 ceph-mon[27862]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:05.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:05 smithi137 ceph-mon[32439]: pgmap v180: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:07 smithi138 ceph-mon[32875]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:07 smithi094 ceph-mon[27862]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:07 smithi137 ceph-mon[32439]: pgmap v181: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:09 smithi138 ceph-mon[32875]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:09.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:09 smithi094 ceph-mon[27862]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:09 smithi137 ceph-mon[32439]: pgmap v182: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:11.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:11 smithi138 ceph-mon[32875]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:11.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:11 smithi094 ceph-mon[27862]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:11 smithi137 ceph-mon[32439]: pgmap v183: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:13 smithi138 ceph-mon[32875]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:13.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:13 smithi094 ceph-mon[27862]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:13 smithi137 ceph-mon[32439]: pgmap v184: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:15.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:15 smithi138 ceph-mon[32875]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:15.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:15 smithi094 ceph-mon[27862]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:15.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:15 smithi137 ceph-mon[32439]: pgmap v185: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-25T15:08:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:17 smithi138 ceph-mon[32875]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:17 smithi094 ceph-mon[27862]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:17 smithi137 ceph-mon[32439]: pgmap v186: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:08:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:19 smithi138 ceph-mon[32875]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:19.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:19 smithi094 ceph-mon[27862]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:19 smithi137 ceph-mon[32439]: pgmap v187: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:21 smithi138 ceph-mon[32875]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:21 smithi094 ceph-mon[27862]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:21.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:21 smithi137 ceph-mon[32439]: pgmap v188: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-04-25T15:08:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:23 smithi138 ceph-mon[32875]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:08:23.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:23 smithi094 ceph-mon[27862]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:08:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:23 smithi137 ceph-mon[32439]: pgmap v189: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-25T15:08:25.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:25 smithi138 ceph-mon[32875]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:08:25.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:25 smithi094 ceph-mon[27862]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:08:25.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:25 smithi137 ceph-mon[32439]: pgmap v190: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:08:26.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:26 smithi138 ceph-mon[32875]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:08:26.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:26 smithi094 ceph-mon[27862]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:08:26.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:26 smithi137 ceph-mon[32439]: pgmap v191: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:08:28.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:28 smithi138 ceph-mon[32875]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:28.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:28 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:08:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:28 smithi094 ceph-mon[27862]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:28.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:28 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:08:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:28 smithi137 ceph-mon[32439]: pgmap v192: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:28 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:08:29.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:29 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:08:29.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:29 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:08:29.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:29 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:08:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:29 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:08:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:29 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:08:29.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:29 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:08:29.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:29 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:08:29.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:29 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:08:29.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:29 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:08:31.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:31 smithi138 ceph-mon[32875]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:31.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:31 smithi094 ceph-mon[27862]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:31.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:31 smithi137 ceph-mon[32439]: pgmap v193: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:33.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:33 smithi138 ceph-mon[32875]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:33.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:33 smithi094 ceph-mon[27862]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:33.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:33 smithi137 ceph-mon[32439]: pgmap v194: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:35.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:35 smithi138 ceph-mon[32875]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:35.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:35 smithi094 ceph-mon[27862]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:35.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:35 smithi137 ceph-mon[32439]: pgmap v195: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:37.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:37 smithi138 ceph-mon[32875]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:37.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:37 smithi094 ceph-mon[27862]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:37.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:37 smithi137 ceph-mon[32439]: pgmap v196: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:39.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:39 smithi138 ceph-mon[32875]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:39.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:39 smithi094 ceph-mon[27862]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:39.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:39 smithi137 ceph-mon[32439]: pgmap v197: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:41.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:41 smithi138 ceph-mon[32875]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:41.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:41 smithi094 ceph-mon[27862]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:41.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:41 smithi137 ceph-mon[32439]: pgmap v198: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:43.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:43 smithi138 ceph-mon[32875]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:43.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:43 smithi094 ceph-mon[27862]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:43.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:43 smithi137 ceph-mon[32439]: pgmap v199: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:45.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:45 smithi138 ceph-mon[32875]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:45.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:45 smithi094 ceph-mon[27862]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:45.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:45 smithi137 ceph-mon[32439]: pgmap v200: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:47.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:47 smithi138 ceph-mon[32875]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:47.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:47 smithi094 ceph-mon[27862]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:47.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:47 smithi137 ceph-mon[32439]: pgmap v201: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:49.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:49 smithi138 ceph-mon[32875]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:49.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:49 smithi094 ceph-mon[27862]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:49.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:49 smithi137 ceph-mon[32439]: pgmap v202: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:51.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:51 smithi138 ceph-mon[32875]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:51.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:51 smithi094 ceph-mon[27862]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:51.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:51 smithi137 ceph-mon[32439]: pgmap v203: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:53.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:53 smithi138 ceph-mon[32875]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:53.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:53 smithi094 ceph-mon[27862]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:53.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:53 smithi137 ceph-mon[32439]: pgmap v204: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:55.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:55 smithi138 ceph-mon[32875]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:55.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:55 smithi094 ceph-mon[27862]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:55.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:55 smithi137 ceph-mon[32439]: pgmap v205: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:57 smithi138 ceph-mon[32875]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:57.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:57 smithi094 ceph-mon[27862]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:57 smithi137 ceph-mon[32439]: pgmap v206: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:08:59.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:08:59 smithi138 ceph-mon[32875]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:59.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:08:59 smithi094 ceph-mon[27862]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:08:59.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:08:59 smithi137 ceph-mon[32439]: pgmap v207: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:01.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:01 smithi138 ceph-mon[32875]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:01.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:01 smithi094 ceph-mon[27862]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:01.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:01 smithi137 ceph-mon[32439]: pgmap v208: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:03.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:03 smithi138 ceph-mon[32875]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:09:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:03 smithi094 ceph-mon[27862]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:09:03.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:03 smithi137 ceph-mon[32439]: pgmap v209: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:09:05.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:05 smithi138 ceph-mon[32875]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:05.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:05 smithi094 ceph-mon[27862]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:05.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:05 smithi137 ceph-mon[32439]: pgmap v210: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:07.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:07 smithi138 ceph-mon[32875]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:09:07.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:07 smithi094 ceph-mon[27862]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:09:07.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:07 smithi137 ceph-mon[32439]: pgmap v211: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:09:09.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:09 smithi138 ceph-mon[32875]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:09.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:09 smithi094 ceph-mon[27862]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:09.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:09 smithi137 ceph-mon[32439]: pgmap v212: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:11.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:11 smithi138 ceph-mon[32875]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:11.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:11 smithi138 ceph-mon[32875]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T15:09:11.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:11 smithi094 ceph-mon[27862]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:11.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:11 smithi094 ceph-mon[27862]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T15:09:11.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:11 smithi137 ceph-mon[32439]: pgmap v213: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:09:11.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:11 smithi137 ceph-mon[32439]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T15:09:13.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:13 smithi138 ceph-mon[32875]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:13.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:13 smithi094 ceph-mon[27862]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:13.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:13 smithi137 ceph-mon[32439]: pgmap v215: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:15.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:15 smithi138 ceph-mon[32875]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:15.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:15 smithi094 ceph-mon[27862]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:15.485 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:15 smithi137 ceph-mon[32439]: pgmap v216: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:17.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:17 smithi138 ceph-mon[32875]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:17.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:17 smithi138 ceph-mon[32875]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T15:09:17.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:17 smithi094 ceph-mon[27862]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:17.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:17 smithi094 ceph-mon[27862]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T15:09:17.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:17 smithi137 ceph-mon[32439]: pgmap v217: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:09:17.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:17 smithi137 ceph-mon[32439]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T15:09:19.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:19 smithi138 ceph-mon[32875]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:09:19.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:19 smithi094 ceph-mon[27862]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:09:19.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:19 smithi137 ceph-mon[32439]: pgmap v219: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:09:21.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:21 smithi138 ceph-mon[32875]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 216 B/s wr, 2 op/s 2024-04-25T15:09:21.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:21 smithi094 ceph-mon[27862]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 216 B/s wr, 2 op/s 2024-04-25T15:09:21.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:21 smithi137 ceph-mon[32439]: pgmap v220: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 216 B/s wr, 2 op/s 2024-04-25T15:09:23.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:23 smithi138 ceph-mon[32875]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:09:23.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:23 smithi094 ceph-mon[27862]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:09:23.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:23 smithi137 ceph-mon[32439]: pgmap v221: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:09:25.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:25 smithi138 ceph-mon[32875]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:09:25.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:25 smithi094 ceph-mon[27862]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:09:25.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:25 smithi137 ceph-mon[32439]: pgmap v222: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:09:27.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:27 smithi094 ceph-mon[27862]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:09:27.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:27 smithi137 ceph-mon[32439]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:09:27.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:27 smithi138 ceph-mon[32875]: pgmap v223: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:09:29.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:29 smithi137 ceph-mon[32439]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T15:09:29.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:29 smithi138 ceph-mon[32875]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T15:09:29.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:29 smithi094 ceph-mon[27862]: pgmap v224: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T15:09:30.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:30 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:09:30.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:30 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:09:30.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:30 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:09:31.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:31 smithi137 ceph-mon[32439]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:09:31.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:31 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:09:31.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:31 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:09:31.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:31 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:09:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:31 smithi138 ceph-mon[32875]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:09:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:31 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:09:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:31 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:09:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:31 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:09:31.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:31 smithi094 ceph-mon[27862]: pgmap v225: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:09:31.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:31 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:09:31.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:31 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:09:31.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:31 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:09:33.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:33 smithi137 ceph-mon[32439]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:09:33.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:33 smithi138 ceph-mon[32875]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:09:33.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:33 smithi094 ceph-mon[27862]: pgmap v226: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:09:35.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:35 smithi138 ceph-mon[32875]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:35 smithi094 ceph-mon[27862]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:35.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:35 smithi137 ceph-mon[32439]: pgmap v227: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:37.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:37 smithi138 ceph-mon[32875]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:37.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:37 smithi094 ceph-mon[27862]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:37.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:37 smithi137 ceph-mon[32439]: pgmap v228: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:39.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:39 smithi138 ceph-mon[32875]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:39.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:39 smithi094 ceph-mon[27862]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:39.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:39 smithi137 ceph-mon[32439]: pgmap v229: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:41 smithi138 ceph-mon[32875]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:41.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:41 smithi094 ceph-mon[27862]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:41.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:41 smithi137 ceph-mon[32439]: pgmap v230: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:43.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:43 smithi094 ceph-mon[27862]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:43 smithi138 ceph-mon[32875]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:43 smithi137 ceph-mon[32439]: pgmap v231: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:45.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:45 smithi138 ceph-mon[32875]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:45 smithi094 ceph-mon[27862]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:45.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:45 smithi137 ceph-mon[32439]: pgmap v232: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:47.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:47 smithi138 ceph-mon[32875]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:47.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:47 smithi094 ceph-mon[27862]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:47.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:47 smithi137 ceph-mon[32439]: pgmap v233: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:49.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:49 smithi138 ceph-mon[32875]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:49.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:49 smithi094 ceph-mon[27862]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:49.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:49 smithi137 ceph-mon[32439]: pgmap v234: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:51.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:51 smithi138 ceph-mon[32875]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:51.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:51 smithi094 ceph-mon[27862]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:51.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:51 smithi137 ceph-mon[32439]: pgmap v235: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:53.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:53 smithi138 ceph-mon[32875]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:53 smithi094 ceph-mon[27862]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:53.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:53 smithi137 ceph-mon[32439]: pgmap v236: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:55 smithi138 ceph-mon[32875]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:55.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:55 smithi094 ceph-mon[27862]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:55.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:55 smithi137 ceph-mon[32439]: pgmap v237: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:57.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:57 smithi138 ceph-mon[32875]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:57.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:57 smithi094 ceph-mon[27862]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:57.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:57 smithi137 ceph-mon[32439]: pgmap v238: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:09:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:09:59 smithi138 ceph-mon[32875]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:59.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:09:59 smithi094 ceph-mon[27862]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:09:59.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:09:59 smithi137 ceph-mon[32439]: pgmap v239: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:00 smithi138 ceph-mon[32875]: overall HEALTH_OK 2024-04-25T15:10:00.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:00 smithi094 ceph-mon[27862]: overall HEALTH_OK 2024-04-25T15:10:00.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:00 smithi137 ceph-mon[32439]: overall HEALTH_OK 2024-04-25T15:10:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:01 smithi138 ceph-mon[32875]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:01 smithi094 ceph-mon[27862]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:01.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:01 smithi137 ceph-mon[32439]: pgmap v240: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:03.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:03 smithi138 ceph-mon[32875]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:03.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:03 smithi094 ceph-mon[27862]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:03.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:03 smithi137 ceph-mon[32439]: pgmap v241: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:05.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:05 smithi138 ceph-mon[32875]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:05.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:05 smithi094 ceph-mon[27862]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:05.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:05 smithi137 ceph-mon[32439]: pgmap v242: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-25T15:10:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:07 smithi138 ceph-mon[32875]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:10:07.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:07 smithi094 ceph-mon[27862]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:10:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:07 smithi137 ceph-mon[32439]: pgmap v243: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:10:09.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:09 smithi138 ceph-mon[32875]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:10:09.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:09 smithi094 ceph-mon[27862]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:10:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:09 smithi137 ceph-mon[32439]: pgmap v244: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:10:11.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:11 smithi138 ceph-mon[32875]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:10:11.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:11 smithi138 ceph-mon[32875]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T15:10:11.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:11 smithi094 ceph-mon[27862]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:10:11.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:11 smithi094 ceph-mon[27862]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T15:10:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:11 smithi137 ceph-mon[32439]: pgmap v245: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:10:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:11 smithi137 ceph-mon[32439]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T15:10:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:13 smithi138 ceph-mon[32875]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 27 KiB/s wr, 4 op/s 2024-04-25T15:10:13.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:13 smithi094 ceph-mon[27862]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 27 KiB/s wr, 4 op/s 2024-04-25T15:10:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:13 smithi137 ceph-mon[32439]: pgmap v247: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 27 KiB/s wr, 4 op/s 2024-04-25T15:10:15.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:15 smithi138 ceph-mon[32875]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 27 KiB/s wr, 4 op/s 2024-04-25T15:10:15.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:15 smithi094 ceph-mon[27862]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 27 KiB/s wr, 4 op/s 2024-04-25T15:10:15.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:15 smithi137 ceph-mon[32439]: pgmap v248: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 27 KiB/s wr, 4 op/s 2024-04-25T15:10:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:17 smithi138 ceph-mon[32875]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:10:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:17 smithi138 ceph-mon[32875]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T15:10:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:17 smithi094 ceph-mon[27862]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:10:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:17 smithi094 ceph-mon[27862]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T15:10:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:17 smithi137 ceph-mon[32439]: pgmap v249: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:10:17.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:17 smithi137 ceph-mon[32439]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T15:10:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:19 smithi138 ceph-mon[32875]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:10:19.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:19 smithi094 ceph-mon[27862]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:10:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:19 smithi137 ceph-mon[32439]: pgmap v251: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:10:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:21 smithi138 ceph-mon[32875]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 219 B/s wr, 2 op/s 2024-04-25T15:10:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:21 smithi094 ceph-mon[27862]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 219 B/s wr, 2 op/s 2024-04-25T15:10:21.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:21 smithi137 ceph-mon[32439]: pgmap v252: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 219 B/s wr, 2 op/s 2024-04-25T15:10:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:23 smithi138 ceph-mon[32875]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:10:23.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:23 smithi094 ceph-mon[27862]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:10:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:23 smithi137 ceph-mon[32439]: pgmap v253: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:10:24.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:24 smithi138 ceph-mon[32875]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:10:24.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:24 smithi094 ceph-mon[27862]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:10:24.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:24 smithi137 ceph-mon[32439]: pgmap v254: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:10:26.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:26 smithi137 ceph-mon[32439]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:10:27.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:26 smithi138 ceph-mon[32875]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:10:27.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:26 smithi094 ceph-mon[27862]: pgmap v255: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T15:10:28.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:28 smithi137 ceph-mon[32439]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T15:10:29.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:28 smithi138 ceph-mon[32875]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T15:10:29.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:28 smithi094 ceph-mon[27862]: pgmap v256: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-25T15:10:30.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:30 smithi137 ceph-mon[32439]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:10:31.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:30 smithi138 ceph-mon[32875]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:10:31.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:30 smithi094 ceph-mon[27862]: pgmap v257: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-25T15:10:32.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:31 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:10:32.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:31 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:10:32.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:31 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:10:33.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:32 smithi138 ceph-mon[32875]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:10:33.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:32 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:10:33.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:32 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:10:33.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:32 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:10:33.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:32 smithi094 ceph-mon[27862]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:10:33.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:32 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:10:33.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:32 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:10:33.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:32 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:10:33.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:32 smithi137 ceph-mon[32439]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-25T15:10:33.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:32 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:10:33.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:32 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:10:33.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:32 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:10:35.034 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:34 smithi137 ceph-mon[32439]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:35.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:34 smithi138 ceph-mon[32875]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:35.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:34 smithi094 ceph-mon[27862]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:37.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:36 smithi138 ceph-mon[32875]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:37.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:36 smithi094 ceph-mon[27862]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:37.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:36 smithi137 ceph-mon[32439]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:39.009 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:38 smithi138 ceph-mon[32875]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:39.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:38 smithi094 ceph-mon[27862]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:39.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:38 smithi137 ceph-mon[32439]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:41.204 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:40 smithi138 ceph-mon[32875]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:10:41.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:40 smithi094 ceph-mon[27862]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:10:41.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:40 smithi137 ceph-mon[32439]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-25T15:10:43.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:42 smithi138 ceph-mon[32875]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:43.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:42 smithi094 ceph-mon[27862]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:43.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:42 smithi137 ceph-mon[32439]: pgmap v263: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:45.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:44 smithi138 ceph-mon[32875]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:45.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:44 smithi094 ceph-mon[27862]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:45.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:44 smithi137 ceph-mon[32439]: pgmap v264: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:47.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:46 smithi138 ceph-mon[32875]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:47.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:46 smithi094 ceph-mon[27862]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:47.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:46 smithi137 ceph-mon[32439]: pgmap v265: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:49.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:49 smithi138 ceph-mon[32875]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:49.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:49 smithi094 ceph-mon[27862]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:49.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:49 smithi137 ceph-mon[32439]: pgmap v266: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:51.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:51 smithi138 ceph-mon[32875]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:51.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:51 smithi094 ceph-mon[27862]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:51.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:51 smithi137 ceph-mon[32439]: pgmap v267: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:53.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:53 smithi094 ceph-mon[27862]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:53.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:53 smithi137 ceph-mon[32439]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:53.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:53 smithi138 ceph-mon[32875]: pgmap v268: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:55.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:55 smithi137 ceph-mon[32439]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:55 smithi138 ceph-mon[32875]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:55.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:55 smithi094 ceph-mon[27862]: pgmap v269: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:57.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:57 smithi137 ceph-mon[32439]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:57.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:57 smithi138 ceph-mon[32875]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:57.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:57 smithi094 ceph-mon[27862]: pgmap v270: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:10:59.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:10:59 smithi137 ceph-mon[32439]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:10:59 smithi138 ceph-mon[32875]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:10:59.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:10:59 smithi094 ceph-mon[27862]: pgmap v271: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:01.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:01 smithi137 ceph-mon[32439]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:01 smithi138 ceph-mon[32875]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:01 smithi094 ceph-mon[27862]: pgmap v272: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:03.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:03 smithi138 ceph-mon[32875]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:11:03.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:03 smithi094 ceph-mon[27862]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:11:03.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:03 smithi137 ceph-mon[32439]: pgmap v273: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T15:11:05.534 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:05 smithi137 ceph-mon[32439]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-25T15:11:05.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:05 smithi138 ceph-mon[32875]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-25T15:11:05.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:05 smithi094 ceph-mon[27862]: pgmap v274: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-25T15:11:07.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:07 smithi138 ceph-mon[32875]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:11:07.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:07 smithi094 ceph-mon[27862]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:11:07.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:07 smithi137 ceph-mon[32439]: pgmap v275: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:11:09.509 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:09 smithi138 ceph-mon[32875]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:09.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:09 smithi094 ceph-mon[27862]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:09.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:09 smithi137 ceph-mon[32439]: pgmap v276: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:11.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:11 smithi138 ceph-mon[32875]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:11.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:11 smithi138 ceph-mon[32875]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T15:11:11.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:11 smithi094 ceph-mon[27862]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:11.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:11 smithi094 ceph-mon[27862]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T15:11:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:11 smithi137 ceph-mon[32439]: pgmap v277: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:11:11.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:11 smithi137 ceph-mon[32439]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T15:11:13.517 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:13 smithi094 ceph-mon[27862]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T15:11:13.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:13 smithi138 ceph-mon[32875]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T15:11:13.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:13 smithi137 ceph-mon[32439]: pgmap v279: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T15:11:15.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:15 smithi138 ceph-mon[32875]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T15:11:15.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:15 smithi094 ceph-mon[27862]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T15:11:15.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:15 smithi137 ceph-mon[32439]: pgmap v280: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 53 KiB/s wr, 20 op/s 2024-04-25T15:11:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:17 smithi138 ceph-mon[32875]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:11:17.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:17 smithi138 ceph-mon[32875]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T15:11:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:17 smithi094 ceph-mon[27862]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:11:17.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:17 smithi094 ceph-mon[27862]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T15:11:17.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:17 smithi137 ceph-mon[32439]: pgmap v281: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-25T15:11:17.734 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:17 smithi137 ceph-mon[32439]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T15:11:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:19 smithi138 ceph-mon[32875]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:11:19.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:19 smithi094 ceph-mon[27862]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:11:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:19 smithi137 ceph-mon[32439]: pgmap v283: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-25T15:11:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:21 smithi138 ceph-mon[32875]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 217 B/s wr, 2 op/s 2024-04-25T15:11:21.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:21 smithi094 ceph-mon[27862]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 217 B/s wr, 2 op/s 2024-04-25T15:11:21.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:21 smithi137 ceph-mon[32439]: pgmap v284: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 217 B/s wr, 2 op/s 2024-04-25T15:11:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:23 smithi138 ceph-mon[32875]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:23.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:23 smithi094 ceph-mon[27862]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:23 smithi137 ceph-mon[32439]: pgmap v285: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:25.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:25 smithi138 ceph-mon[32875]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:25.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:25 smithi094 ceph-mon[27862]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:25.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:25 smithi137 ceph-mon[32439]: pgmap v286: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-25T15:11:27.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:27 smithi138 ceph-mon[32875]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:11:27.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:27 smithi094 ceph-mon[27862]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:11:27.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:27 smithi137 ceph-mon[32439]: pgmap v287: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:11:29.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:29 smithi138 ceph-mon[32875]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-25T15:11:29.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:29 smithi094 ceph-mon[27862]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-25T15:11:29.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:29 smithi137 ceph-mon[32439]: pgmap v288: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-25T15:11:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:31 smithi138 ceph-mon[32875]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:31.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:31 smithi094 ceph-mon[27862]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:31.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:31 smithi137 ceph-mon[32439]: pgmap v289: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:33.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:33 smithi138 ceph-mon[32875]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:33.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:33 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:11:33.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:33 smithi094 ceph-mon[27862]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:33.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:33 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:11:33.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:33 smithi137 ceph-mon[32439]: pgmap v290: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:33.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:33 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:11:34.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:34 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:11:34.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:34 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:11:34.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:34 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:11:34.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:34 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:11:34.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:34 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:11:34.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:34 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:11:34.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:34 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:11:34.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:34 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:11:34.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:34 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:11:35.633 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:35 smithi137 ceph-mon[32439]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:35.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:35 smithi138 ceph-mon[32875]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:35 smithi094 ceph-mon[27862]: pgmap v291: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:37.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:37 smithi138 ceph-mon[32875]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:37.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:37 smithi094 ceph-mon[27862]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:37.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:37 smithi137 ceph-mon[32439]: pgmap v292: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:39.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:39 smithi138 ceph-mon[32875]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:39.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:39 smithi094 ceph-mon[27862]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:39.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:39 smithi137 ceph-mon[32439]: pgmap v293: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:41.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:41 smithi138 ceph-mon[32875]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:41.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:41 smithi094 ceph-mon[27862]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:41.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:41 smithi137 ceph-mon[32439]: pgmap v294: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:43.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:43 smithi138 ceph-mon[32875]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:43.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:43 smithi094 ceph-mon[27862]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:43.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:43 smithi137 ceph-mon[32439]: pgmap v295: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:45.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:45 smithi138 ceph-mon[32875]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:45.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:45 smithi094 ceph-mon[27862]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:45.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:45 smithi137 ceph-mon[32439]: pgmap v296: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:47.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:47 smithi138 ceph-mon[32875]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:47.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:47 smithi094 ceph-mon[27862]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:47.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:47 smithi137 ceph-mon[32439]: pgmap v297: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:49.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:49 smithi138 ceph-mon[32875]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:49.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:49 smithi094 ceph-mon[27862]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:49.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:49 smithi137 ceph-mon[32439]: pgmap v298: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:51.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:51 smithi138 ceph-mon[32875]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:51.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:51 smithi094 ceph-mon[27862]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:51.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:51 smithi137 ceph-mon[32439]: pgmap v299: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:53.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:53 smithi138 ceph-mon[32875]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:53.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:53 smithi094 ceph-mon[27862]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:53.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:53 smithi137 ceph-mon[32439]: pgmap v300: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:55 smithi138 ceph-mon[32875]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:55.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:55 smithi094 ceph-mon[27862]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:55.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:55 smithi137 ceph-mon[32439]: pgmap v301: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:57.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:57 smithi138 ceph-mon[32875]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:57.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:57 smithi094 ceph-mon[27862]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:57.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:57 smithi137 ceph-mon[32439]: pgmap v302: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:11:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:11:59 smithi138 ceph-mon[32875]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:59.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:11:59 smithi094 ceph-mon[27862]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:11:59.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:11:59 smithi137 ceph-mon[32439]: pgmap v303: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:01.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:01 smithi138 ceph-mon[32875]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:01.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:01 smithi094 ceph-mon[27862]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:01.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:01 smithi137 ceph-mon[32439]: pgmap v304: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:02.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:02 smithi138 ceph-mon[32875]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:02.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:02 smithi094 ceph-mon[27862]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:02.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:02 smithi137 ceph-mon[32439]: pgmap v305: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:05.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:04 smithi138 ceph-mon[32875]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:05.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:04 smithi094 ceph-mon[27862]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:05.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:04 smithi137 ceph-mon[32439]: pgmap v306: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:07.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:06 smithi138 ceph-mon[32875]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:12:07.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:06 smithi094 ceph-mon[27862]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:12:07.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:06 smithi137 ceph-mon[32439]: pgmap v307: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-25T15:12:09.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:08 smithi138 ceph-mon[32875]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:09.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:08 smithi094 ceph-mon[27862]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:09.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:08 smithi137 ceph-mon[32439]: pgmap v308: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:11.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:10 smithi138 ceph-mon[32875]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:11.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:10 smithi094 ceph-mon[27862]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:11.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:10 smithi137 ceph-mon[32439]: pgmap v309: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-25T15:12:12.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:11 smithi138 ceph-mon[32875]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T15:12:12.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:11 smithi094 ceph-mon[27862]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T15:12:12.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:11 smithi137 ceph-mon[32439]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T15:12:13.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T15:12:13.096 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:12 smithi094 ceph-mon[27862]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:13.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:12 smithi138 ceph-mon[32875]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:13.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.225+0000 7fd337e00640 1 -- 172.21.15.94:0/2409578937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 msgr2=0x7fd33810e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.225+0000 7fd337e00640 1 --2- 172.21.15.94:0/2409578937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 0x7fd33810e100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd324009900 tx=0x7fd32402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.225+0000 7fd337e00640 1 -- 172.21.15.94:0/2409578937 shutdown_connections 2024-04-25T15:12:13.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.225+0000 7fd337e00640 1 --2- 172.21.15.94:0/2409578937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd33810e6d0 0x7fd338110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.225+0000 7fd337e00640 1 --2- 172.21.15.94:0/2409578937 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 0x7fd33810e100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.225+0000 7fd337e00640 1 --2- 172.21.15.94:0/2409578937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd338100b20 0x7fd338100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.225+0000 7fd337e00640 1 -- 172.21.15.94:0/2409578937 >> 172.21.15.94:0/2409578937 conn(0x7fd3380fc770 msgr2=0x7fd3380feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:13.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.226+0000 7fd337e00640 1 -- 172.21.15.94:0/2409578937 shutdown_connections 2024-04-25T15:12:13.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.226+0000 7fd337e00640 1 -- 172.21.15.94:0/2409578937 wait complete. 2024-04-25T15:12:13.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.226+0000 7fd337e00640 1 Processor -- start 2024-04-25T15:12:13.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337e00640 1 -- start start 2024-04-25T15:12:13.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd338100b20 0x7fd3381a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 0x7fd3381a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33810e6d0 0x7fd33819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd338114250 con 0x7fd338100b20 2024-04-25T15:12:13.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd3381140d0 con 0x7fd3381014d0 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd3381143d0 con 0x7fd33810e6d0 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd336a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd338100b20 0x7fd3381a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd336000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 0x7fd3381a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd336a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd338100b20 0x7fd3381a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54170/0 (socket says 172.21.15.94:54170) 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd336a00640 1 -- 172.21.15.94:0/1529375240 learned_addr learned my addr 172.21.15.94:0/1529375240 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.227+0000 7fd337400640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33810e6d0 0x7fd33819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd336a00640 1 -- 172.21.15.94:0/1529375240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33810e6d0 msgr2=0x7fd33819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd336a00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33810e6d0 0x7fd33819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd336a00640 1 -- 172.21.15.94:0/1529375240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 msgr2=0x7fd3381a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd336a00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 0x7fd3381a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd336a00640 1 -- 172.21.15.94:0/1529375240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33819ad90 con 0x7fd338100b20 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd336a00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd338100b20 0x7fd3381a0460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd32800e9f0 tx=0x7fd32800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd32f400640 1 -- 172.21.15.94:0/1529375240 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32800ccc0 con 0x7fd338100b20 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33819b080 con 0x7fd338100b20 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd32f400640 1 -- 172.21.15.94:0/1529375240 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3280108e0 con 0x7fd338100b20 2024-04-25T15:12:13.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.228+0000 7fd32f400640 1 -- 172.21.15.94:0/1529375240 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328018880 con 0x7fd338100b20 2024-04-25T15:12:13.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.229+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3381a7320 con 0x7fd338100b20 2024-04-25T15:12:13.234 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:12 smithi137 ceph-mon[32439]: pgmap v311: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:13.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.230+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd338050530 con 0x7fd338100b20 2024-04-25T15:12:13.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.232+0000 7fd32f400640 1 -- 172.21.15.94:0/1529375240 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd328010430 con 0x7fd338100b20 2024-04-25T15:12:13.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.233+0000 7fd32f400640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd31c0a3d90 0x7fd31c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.233+0000 7fd32f400640 1 -- 172.21.15.94:0/1529375240 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7fd3280d0bf0 con 0x7fd338100b20 2024-04-25T15:12:13.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.234+0000 7fd336000640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd31c0a3d90 0x7fd31c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.234+0000 7fd336000640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd31c0a3d90 0x7fd31c0a6250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd324005ab0 tx=0x7fd324005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:13.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.239+0000 7fd32f400640 1 -- 172.21.15.94:0/1529375240 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd32809d1f0 con 0x7fd338100b20 2024-04-25T15:12:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.468+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd338062390 con 0x7fd31c0a3d90 2024-04-25T15:12:13.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.478+0000 7fd32f400640 1 -- 172.21.15.94:0/1529375240 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fd338062390 con 0x7fd31c0a3d90 2024-04-25T15:12:13.480 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T15:02:10", "first": "2024-04-25T15:03:00", "last": "2024-04-25T15:12:00", "last_pruned": "2024-04-25T15:12:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T15:12:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd31c0a3d90 msgr2=0x7fd31c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd31c0a3d90 0x7fd31c0a6250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd324005ab0 tx=0x7fd324005a20 comp rx=0 tx=0).stop 2024-04-25T15:12:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd338100b20 msgr2=0x7fd3381a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd338100b20 0x7fd3381a0460 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd32800e9f0 tx=0x7fd32800edf0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 shutdown_connections 2024-04-25T15:12:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd31c0a3d90 0x7fd31c0a6250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd33810e6d0 0x7fd33819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd3381014d0 0x7fd3381a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 --2- 172.21.15.94:0/1529375240 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd338100b20 0x7fd3381a0460 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 >> 172.21.15.94:0/1529375240 conn(0x7fd3380fc770 msgr2=0x7fd3380feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 shutdown_connections 2024-04-25T15:12:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.481+0000 7fd337e00640 1 -- 172.21.15.94:0/1529375240 wait complete. 2024-04-25T15:12:13.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T15:12:13.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.682+0000 7fa7ac400640 1 -- 172.21.15.94:0/4187459930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a4102b80 msgr2=0x7fa7a4106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.682+0000 7fa7ac400640 1 --2- 172.21.15.94:0/4187459930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a4102b80 0x7fa7a4106bc0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa790009900 tx=0x7fa79002a3f0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.682+0000 7fa7ac400640 1 -- 172.21.15.94:0/4187459930 shutdown_connections 2024-04-25T15:12:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.682+0000 7fa7ac400640 1 --2- 172.21.15.94:0/4187459930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41072a0 0x7fa7a410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.682+0000 7fa7ac400640 1 --2- 172.21.15.94:0/4187459930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a4102b80 0x7fa7a4106bc0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.682+0000 7fa7ac400640 1 --2- 172.21.15.94:0/4187459930 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a41021d0 0x7fa7a41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.682+0000 7fa7ac400640 1 -- 172.21.15.94:0/4187459930 >> 172.21.15.94:0/4187459930 conn(0x7fa7a40793e0 msgr2=0x7fa7a40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:13.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.683+0000 7fa7ac400640 1 -- 172.21.15.94:0/4187459930 shutdown_connections 2024-04-25T15:12:13.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.683+0000 7fa7ac400640 1 -- 172.21.15.94:0/4187459930 wait complete. 2024-04-25T15:12:13.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 Processor -- start 2024-04-25T15:12:13.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 -- start start 2024-04-25T15:12:13.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41021d0 0x7fa7a41a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a41072a0 0x7fa7a41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a419a780 0x7fa7a419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4112210 con 0x7fa7a41021d0 2024-04-25T15:12:13.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4112090 con 0x7fa7a419a780 2024-04-25T15:12:13.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7ac400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a4112390 con 0x7fa7a41072a0 2024-04-25T15:12:13.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7a9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41021d0 0x7fa7a41a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.684+0000 7fa7a9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a41072a0 0x7fa7a41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a41072a0 0x7fa7a41a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40090/0 (socket says 172.21.15.94:40090) 2024-04-25T15:12:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41021d0 0x7fa7a41a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54180/0 (socket says 172.21.15.94:54180) 2024-04-25T15:12:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 -- 172.21.15.94:0/1163074371 learned_addr learned my addr 172.21.15.94:0/1163074371 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7aa400640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a419a780 0x7fa7a419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 -- 172.21.15.94:0/1163074371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a419a780 msgr2=0x7fa7a419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a419a780 0x7fa7a419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 -- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41021d0 msgr2=0x7fa7a41a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41021d0 0x7fa7a41a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 -- 172.21.15.94:0/1163074371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a419b310 con 0x7fa7a41072a0 2024-04-25T15:12:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9a00640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41021d0 0x7fa7a41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:12:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.685+0000 7fa7a9000640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a41072a0 0x7fa7a41a0b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa790005ab0 tx=0x7fa790003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:13.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.686+0000 7fa7a2000640 1 -- 172.21.15.94:0/1163074371 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79003f070 con 0x7fa7a41072a0 2024-04-25T15:12:13.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.686+0000 7fa7a2000640 1 -- 172.21.15.94:0/1163074371 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa790031b50 con 0x7fa7a41072a0 2024-04-25T15:12:13.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.686+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a419b5a0 con 0x7fa7a41072a0 2024-04-25T15:12:13.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.686+0000 7fa7a2000640 1 -- 172.21.15.94:0/1163074371 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790035030 con 0x7fa7a41072a0 2024-04-25T15:12:13.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.686+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a41a7780 con 0x7fa7a41072a0 2024-04-25T15:12:13.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.687+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a41036a0 con 0x7fa7a41072a0 2024-04-25T15:12:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.691+0000 7fa7a2000640 1 -- 172.21.15.94:0/1163074371 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa790042070 con 0x7fa7a41072a0 2024-04-25T15:12:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.693+0000 7fa7a2000640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa78c0a3e40 0x7fa78c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.693+0000 7fa7a2000640 1 -- 172.21.15.94:0/1163074371 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7fa7900f1610 con 0x7fa7a41072a0 2024-04-25T15:12:13.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.693+0000 7fa7a9a00640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa78c0a3e40 0x7fa78c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:13.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.693+0000 7fa7a2000640 1 -- 172.21.15.94:0/1163074371 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7900bd5c0 con 0x7fa7a41072a0 2024-04-25T15:12:13.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.694+0000 7fa7a9a00640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa78c0a3e40 0x7fa78c0a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa798009fc0 tx=0x7fa798005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:13.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.916+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 --> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa7a4102b80 con 0x7fa78c0a3e40 2024-04-25T15:12:13.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.929+0000 7fa7a2000640 1 -- 172.21.15.94:0/1163074371 <== mgr.15027 v2:172.21.15.94:6800/2411855535 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa7a4102b80 con 0x7fa78c0a3e40 2024-04-25T15:12:13.934 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-04-25T15:12:13.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa78c0a3e40 msgr2=0x7fa78c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa78c0a3e40 0x7fa78c0a6300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa798009fc0 tx=0x7fa798005f90 comp rx=0 tx=0).stop 2024-04-25T15:12:13.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a41072a0 msgr2=0x7fa7a41a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:13.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a41072a0 0x7fa7a41a0b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa790005ab0 tx=0x7fa790003b90 comp rx=0 tx=0).stop 2024-04-25T15:12:13.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 shutdown_connections 2024-04-25T15:12:13.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fa78c0a3e40 0x7fa78c0a6300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fa7a419a780 0x7fa7a419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7a41072a0 0x7fa7a41a0b50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 --2- 172.21.15.94:0/1163074371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a41021d0 0x7fa7a41a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:13.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 >> 172.21.15.94:0/1163074371 conn(0x7fa7a40793e0 msgr2=0x7fa7a40ffba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:13.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 shutdown_connections 2024-04-25T15:12:13.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:13.933+0000 7fa7ac400640 1 -- 172.21.15.94:0/1163074371 wait complete. 2024-04-25T15:12:13.960 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T15:12:13.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:12:14.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 -- 172.21.15.94:0/2754879895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd408101fe0 msgr2=0x7fd4081023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 --2- 172.21.15.94:0/2754879895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd408101fe0 0x7fd4081023c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8009900 tx=0x7fd3f802a3f0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 -- 172.21.15.94:0/2754879895 shutdown_connections 2024-04-25T15:12:14.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 --2- 172.21.15.94:0/2754879895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4081070b0 0x7fd40810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 --2- 172.21.15.94:0/2754879895 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd408102990 0x7fd4081069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 --2- 172.21.15.94:0/2754879895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd408101fe0 0x7fd4081023c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 -- 172.21.15.94:0/2754879895 >> 172.21.15.94:0/2754879895 conn(0x7fd4080fd880 msgr2=0x7fd4080ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 -- 172.21.15.94:0/2754879895 shutdown_connections 2024-04-25T15:12:14.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.165+0000 7fd407200640 1 -- 172.21.15.94:0/2754879895 wait complete. 2024-04-25T15:12:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 Processor -- start 2024-04-25T15:12:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 -- start start 2024-04-25T15:12:14.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd408101fe0 0x7fd4081a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4081070b0 0x7fd4081a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40819a5d0 0x7fd40819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fd408111ff0 con 0x7fd4081070b0 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fd408111e70 con 0x7fd40819a5d0 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.166+0000 7fd407200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd408112170 con 0x7fd408101fe0 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4081070b0 0x7fd4081a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4081070b0 0x7fd4081a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54204/0 (socket says 172.21.15.94:54204) 2024-04-25T15:12:14.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 -- 172.21.15.94:0/196752024 learned_addr learned my addr 172.21.15.94:0/196752024 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd406800640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40819a5d0 0x7fd40819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405e00640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd408101fe0 0x7fd4081a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 -- 172.21.15.94:0/196752024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd408101fe0 msgr2=0x7fd4081a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd408101fe0 0x7fd4081a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 -- 172.21.15.94:0/196752024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40819a5d0 msgr2=0x7fd40819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40819a5d0 0x7fd40819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 -- 172.21.15.94:0/196752024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40819b0f0 con 0x7fd4081070b0 2024-04-25T15:12:14.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd405400640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4081070b0 0x7fd4081a0960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400e9c0 tx=0x7fd3f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.167+0000 7fd3fea00640 1 -- 172.21.15.94:0/196752024 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f400cd80 con 0x7fd4081070b0 2024-04-25T15:12:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.168+0000 7fd3fea00640 1 -- 172.21.15.94:0/196752024 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3f40054d0 con 0x7fd4081070b0 2024-04-25T15:12:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.168+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40819b3e0 con 0x7fd4081070b0 2024-04-25T15:12:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.168+0000 7fd3fea00640 1 -- 172.21.15.94:0/196752024 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4010430 con 0x7fd4081070b0 2024-04-25T15:12:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.168+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081a7630 con 0x7fd4081070b0 2024-04-25T15:12:14.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.169+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd408068c90 con 0x7fd4081070b0 2024-04-25T15:12:14.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.171+0000 7fd3fea00640 1 -- 172.21.15.94:0/196752024 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd3f4004fd0 con 0x7fd4081070b0 2024-04-25T15:12:14.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.172+0000 7fd3fea00640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd3ec0a3d20 0x7fd3ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.173+0000 7fd3fea00640 1 -- 172.21.15.94:0/196752024 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7fd3f40d0d50 con 0x7fd4081070b0 2024-04-25T15:12:14.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.173+0000 7fd405e00640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd3ec0a3d20 0x7fd3ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.174+0000 7fd405e00640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd3ec0a3d20 0x7fd3ec0a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8031440 tx=0x7fd3f80095f0 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:14.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.177+0000 7fd3fea00640 1 -- 172.21.15.94:0/196752024 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3f409d350 con 0x7fd4081070b0 2024-04-25T15:12:14.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.374+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd408102a20 con 0x7fd4081070b0 2024-04-25T15:12:14.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.377+0000 7fd3fea00640 1 -- 172.21.15.94:0/196752024 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+23218 (secure 0 0 0) 0x7fd3f409cc50 con 0x7fd4081070b0 2024-04-25T15:12:14.379 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:12:14.379 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":96,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:12:10.775917+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/702744035":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3354212797":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6801/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1994260887":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/2523030980":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6800/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:12:14.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.379+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd3ec0a3d20 msgr2=0x7fd3ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.379+0000 7fd407200640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd3ec0a3d20 0x7fd3ec0a61e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd3f8031440 tx=0x7fd3f80095f0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.379+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4081070b0 msgr2=0x7fd4081a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.379+0000 7fd407200640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4081070b0 0x7fd4081a0960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400e9c0 tx=0x7fd3f400ee80 comp rx=0 tx=0).stop 2024-04-25T15:12:14.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.379+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 shutdown_connections 2024-04-25T15:12:14.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.379+0000 7fd407200640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fd3ec0a3d20 0x7fd3ec0a61e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.379+0000 7fd407200640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fd40819a5d0 0x7fd40819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.380+0000 7fd407200640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4081070b0 0x7fd4081a0960 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.380+0000 7fd407200640 1 --2- 172.21.15.94:0/196752024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd408101fe0 0x7fd4081a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.380+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 >> 172.21.15.94:0/196752024 conn(0x7fd4080fd880 msgr2=0x7fd4080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:14.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.380+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 shutdown_connections 2024-04-25T15:12:14.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.380+0000 7fd407200640 1 -- 172.21.15.94:0/196752024 wait complete. 2024-04-25T15:12:14.399 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:12:14.400 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:12:14.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.590+0000 7ff1ffe00640 1 -- 172.21.15.94:0/2242850519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200102060 msgr2=0x7ff200102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.590+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/2242850519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200102060 0x7ff200102440 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009900 tx=0x7ff1f002a410 comp rx=0 tx=0).stop 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.590+0000 7ff1ffe00640 1 -- 172.21.15.94:0/2242850519 shutdown_connections 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.590+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/2242850519 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff200107130 0x7ff20010ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.590+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/2242850519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff200102a10 0x7ff200106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.590+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/2242850519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200102060 0x7ff200102440 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.590+0000 7ff1ffe00640 1 -- 172.21.15.94:0/2242850519 >> 172.21.15.94:0/2242850519 conn(0x7ff2000fd920 msgr2=0x7ff2000ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.591+0000 7ff1ffe00640 1 -- 172.21.15.94:0/2242850519 shutdown_connections 2024-04-25T15:12:14.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.591+0000 7ff1ffe00640 1 -- 172.21.15.94:0/2242850519 wait complete. 2024-04-25T15:12:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 Processor -- start 2024-04-25T15:12:14.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 -- start start 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff200102060 0x7ff2001a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200107130 0x7ff2001a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff20019a640 0x7ff20019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff2001120c0 con 0x7ff200107130 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff200111f40 con 0x7ff20019a640 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ffe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff200112240 con 0x7ff200102060 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1fe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200107130 0x7ff2001a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1fe000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200107130 0x7ff2001a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54218/0 (socket says 172.21.15.94:54218) 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1fe000640 1 -- 172.21.15.94:0/1621468782 learned_addr learned my addr 172.21.15.94:0/1621468782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1fea00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff200102060 0x7ff2001a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.592+0000 7ff1ff400640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff20019a640 0x7ff20019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1fe000640 1 -- 172.21.15.94:0/1621468782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff200102060 msgr2=0x7ff2001a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1fe000640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff200102060 0x7ff2001a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1fe000640 1 -- 172.21.15.94:0/1621468782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff20019a640 msgr2=0x7ff20019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1fe000640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff20019a640 0x7ff20019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1fe000640 1 -- 172.21.15.94:0/1621468782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20019b240 con 0x7ff200107130 2024-04-25T15:12:14.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1fe000640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200107130 0x7ff2001a0ac0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00e9c0 tx=0x7ff1ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1f7400640 1 -- 172.21.15.94:0/1621468782 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00cd80 con 0x7ff200107130 2024-04-25T15:12:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1f7400640 1 -- 172.21.15.94:0/1621468782 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1ec0054d0 con 0x7ff200107130 2024-04-25T15:12:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.593+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2001a7320 con 0x7ff200107130 2024-04-25T15:12:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.594+0000 7ff1f7400640 1 -- 172.21.15.94:0/1621468782 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec010430 con 0x7ff200107130 2024-04-25T15:12:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.594+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2001a7720 con 0x7ff200107130 2024-04-25T15:12:14.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.595+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff200103530 con 0x7ff200107130 2024-04-25T15:12:14.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.599+0000 7ff1f7400640 1 -- 172.21.15.94:0/1621468782 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1ec004fd0 con 0x7ff200107130 2024-04-25T15:12:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.601+0000 7ff1f7400640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1e40a3d20 0x7ff1e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:14.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.601+0000 7ff1f7400640 1 -- 172.21.15.94:0/1621468782 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7ff1ec0d0ca0 con 0x7ff200107130 2024-04-25T15:12:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.601+0000 7ff1fea00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1e40a3d20 0x7ff1e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:14.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.602+0000 7ff1fea00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1e40a3d20 0x7ff1e40a61e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005ab0 tx=0x7ff1f0005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:14.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.605+0000 7ff1f7400640 1 -- 172.21.15.94:0/1621468782 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1ec09d2a0 con 0x7ff200107130 2024-04-25T15:12:14.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.808+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff200102a10 con 0x7ff200107130 2024-04-25T15:12:14.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.809+0000 7ff1f7400640 1 -- 172.21.15.94:0/1621468782 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7ff1ec09cba0 con 0x7ff200107130 2024-04-25T15:12:14.811 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:12:14.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1e40a3d20 msgr2=0x7ff1e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1e40a3d20 0x7ff1e40a61e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0005ab0 tx=0x7ff1f0005a20 comp rx=0 tx=0).stop 2024-04-25T15:12:14.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200107130 msgr2=0x7ff2001a0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:14.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200107130 0x7ff2001a0ac0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00e9c0 tx=0x7ff1ec00ee80 comp rx=0 tx=0).stop 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 shutdown_connections 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1e40a3d20 0x7ff1e40a61e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff20019a640 0x7ff20019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff200107130 0x7ff2001a0ac0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 --2- 172.21.15.94:0/1621468782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff200102060 0x7ff2001a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 >> 172.21.15.94:0/1621468782 conn(0x7ff2000fd920 msgr2=0x7ff2000fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 shutdown_connections 2024-04-25T15:12:14.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:14.812+0000 7ff1ffe00640 1 -- 172.21.15.94:0/1621468782 wait complete. 2024-04-25T15:12:14.832 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:12:14.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:14 smithi094 ceph-mon[27862]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:12:14.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:14 smithi094 ceph-mon[27862]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:14.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:14 smithi094 ceph-mon[27862]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:12:14.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:14 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/196752024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:15.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 -- 172.21.15.94:0/2892040150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104c10 msgr2=0x7f28d8108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 --2- 172.21.15.94:0/2892040150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104c10 0x7f28d8108c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009900 tx=0x7f28c402a400 comp rx=0 tx=0).stop 2024-04-25T15:12:15.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 -- 172.21.15.94:0/2892040150 shutdown_connections 2024-04-25T15:12:15.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 --2- 172.21.15.94:0/2892040150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d8109330 0x7f28d8110f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 --2- 172.21.15.94:0/2892040150 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104c10 0x7f28d8108c50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 --2- 172.21.15.94:0/2892040150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28d8104260 0x7f28d8104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 -- 172.21.15.94:0/2892040150 >> 172.21.15.94:0/2892040150 conn(0x7f28d80ffa60 msgr2=0x7f28d8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:15.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 -- 172.21.15.94:0/2892040150 shutdown_connections 2024-04-25T15:12:15.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.031+0000 7f28de000640 1 -- 172.21.15.94:0/2892040150 wait complete. 2024-04-25T15:12:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.032+0000 7f28de000640 1 Processor -- start 2024-04-25T15:12:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.032+0000 7f28de000640 1 -- start start 2024-04-25T15:12:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28de000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104260 0x7f28d81a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28de000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28d8109330 0x7f28d81a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28de000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d819a6a0 0x7f28d819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28de000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8114250 con 0x7f28d819a6a0 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28de000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81140d0 con 0x7f28d8104260 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28de000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81143d0 con 0x7f28d8109330 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28dd600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d819a6a0 0x7f28d819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28dd600640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d819a6a0 0x7f28d819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54224/0 (socket says 172.21.15.94:54224) 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28dd600640 1 -- 172.21.15.94:0/112710616 learned_addr learned my addr 172.21.15.94:0/112710616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:15.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28dcc00640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104260 0x7f28d81a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.033+0000 7f28d7e00640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28d8109330 0x7f28d81a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28dd600640 1 -- 172.21.15.94:0/112710616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28d8109330 msgr2=0x7f28d81a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28dd600640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28d8109330 0x7f28d81a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28dd600640 1 -- 172.21.15.94:0/112710616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104260 msgr2=0x7f28d81a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28dd600640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104260 0x7f28d81a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28dd600640 1 -- 172.21.15.94:0/112710616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d819b2a0 con 0x7f28d819a6a0 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28dcc00640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104260 0x7f28d81a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:12:15.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28dd600640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d819a6a0 0x7f28d819ab20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28d000efd0 tx=0x7f28d000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28d5600640 1 -- 172.21.15.94:0/112710616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0003b70 con 0x7f28d819a6a0 2024-04-25T15:12:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28d5600640 1 -- 172.21.15.94:0/112710616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d0003d10 con 0x7f28d819a6a0 2024-04-25T15:12:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d81a7330 con 0x7f28d819a6a0 2024-04-25T15:12:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28d5600640 1 -- 172.21.15.94:0/112710616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d00108a0 con 0x7f28d819a6a0 2024-04-25T15:12:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.034+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81a76e0 con 0x7f28d819a6a0 2024-04-25T15:12:15.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.035+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d8105730 con 0x7f28d819a6a0 2024-04-25T15:12:15.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.037+0000 7f28d5600640 1 -- 172.21.15.94:0/112710616 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f28d0004020 con 0x7f28d819a6a0 2024-04-25T15:12:15.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.039+0000 7f28d5600640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f28c00a3d20 0x7f28c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.039+0000 7f28d5600640 1 -- 172.21.15.94:0/112710616 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f28d00d0be0 con 0x7f28d819a6a0 2024-04-25T15:12:15.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.039+0000 7f28dcc00640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f28c00a3d20 0x7f28c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.040+0000 7f28dcc00640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f28c00a3d20 0x7f28c00a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009fc0 tx=0x7f28cc005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:15.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.043+0000 7f28d5600640 1 -- 172.21.15.94:0/112710616 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28d009d1e0 con 0x7f28d819a6a0 2024-04-25T15:12:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:14 smithi138 ceph-mon[32875]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:12:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:14 smithi138 ceph-mon[32875]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:15.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:14 smithi138 ceph-mon[32875]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:12:15.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:14 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/196752024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:15.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:14 smithi137 ceph-mon[32439]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:12:15.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:14 smithi137 ceph-mon[32439]: pgmap v312: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-04-25T15:12:15.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:14 smithi137 ceph-mon[32439]: from='client.25336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:12:15.233 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:14 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/196752024' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:15.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.231+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f28d810c560 con 0x7f28d819a6a0 2024-04-25T15:12:15.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.232+0000 7f28d5600640 1 -- 172.21.15.94:0/112710616 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f28d009cae0 con 0x7f28d819a6a0 2024-04-25T15:12:15.234 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:12:15.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f28c00a3d20 msgr2=0x7f28c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f28c00a3d20 0x7f28c00a61e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009fc0 tx=0x7f28cc005f90 comp rx=0 tx=0).stop 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d819a6a0 msgr2=0x7f28d819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d819a6a0 0x7f28d819ab20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28d000efd0 tx=0x7f28d000c6a0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 shutdown_connections 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f28c00a3d20 0x7f28c00a61e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28d819a6a0 0x7f28d819ab20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28d8109330 0x7f28d81a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 --2- 172.21.15.94:0/112710616 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f28d8104260 0x7f28d81a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 >> 172.21.15.94:0/112710616 conn(0x7f28d80ffa60 msgr2=0x7f28d8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:15.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 shutdown_connections 2024-04-25T15:12:15.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.235+0000 7f28de000640 1 -- 172.21.15.94:0/112710616 wait complete. 2024-04-25T15:12:15.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:12:15.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 -- 172.21.15.94:0/3536184378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53741038d0 msgr2=0x7f537410bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 --2- 172.21.15.94:0/3536184378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53741038d0 0x7f537410bdb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5360009900 tx=0x7f536002a480 comp rx=0 tx=0).stop 2024-04-25T15:12:15.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 -- 172.21.15.94:0/3536184378 shutdown_connections 2024-04-25T15:12:15.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 --2- 172.21.15.94:0/3536184378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f537410c3d0 0x7f537410e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 --2- 172.21.15.94:0/3536184378 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f53741038d0 0x7f537410bdb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 --2- 172.21.15.94:0/3536184378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5374102fb0 0x7f5374103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 -- 172.21.15.94:0/3536184378 >> 172.21.15.94:0/3536184378 conn(0x7f53740fc7d0 msgr2=0x7f53740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:15.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 -- 172.21.15.94:0/3536184378 shutdown_connections 2024-04-25T15:12:15.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.450+0000 7f537aa00640 1 -- 172.21.15.94:0/3536184378 wait complete. 2024-04-25T15:12:15.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 Processor -- start 2024-04-25T15:12:15.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 -- start start 2024-04-25T15:12:15.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374102fb0 0x7f53741a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53741038d0 0x7f53741a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f537410c3d0 0x7f537419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f5374112060 con 0x7f53741038d0 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f5374111ee0 con 0x7f537410c3d0 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.451+0000 7f537aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53741121e0 con 0x7f5374102fb0 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374102fb0 0x7f53741a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5378c00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53741038d0 0x7f53741a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374102fb0 0x7f53741a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:40150/0 (socket says 172.21.15.94:40150) 2024-04-25T15:12:15.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 -- 172.21.15.94:0/3430302390 learned_addr learned my addr 172.21.15.94:0/3430302390 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:15.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f537a000640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f537410c3d0 0x7f537419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 -- 172.21.15.94:0/3430302390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f537410c3d0 msgr2=0x7f537419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f537410c3d0 0x7f537419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 -- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53741038d0 msgr2=0x7f53741a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53741038d0 0x7f53741a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 -- 172.21.15.94:0/3430302390 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537419ade0 con 0x7f5374102fb0 2024-04-25T15:12:15.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5378c00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53741038d0 0x7f53741a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:12:15.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.452+0000 7f5379600640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374102fb0 0x7f53741a0460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9f0 tx=0x7f536400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:15.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.453+0000 7f5372000640 1 -- 172.21.15.94:0/3430302390 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400cd80 con 0x7f5374102fb0 2024-04-25T15:12:15.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.453+0000 7f5372000640 1 -- 172.21.15.94:0/3430302390 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53640108e0 con 0x7f5374102fb0 2024-04-25T15:12:15.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.453+0000 7f5372000640 1 -- 172.21.15.94:0/3430302390 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364018980 con 0x7f5374102fb0 2024-04-25T15:12:15.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.453+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537419b0d0 con 0x7f5374102fb0 2024-04-25T15:12:15.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.453+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a7320 con 0x7f5374102fb0 2024-04-25T15:12:15.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.454+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5374068c90 con 0x7f5374102fb0 2024-04-25T15:12:15.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.456+0000 7f5372000640 1 -- 172.21.15.94:0/3430302390 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5364010430 con 0x7f5374102fb0 2024-04-25T15:12:15.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.457+0000 7f5372000640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f53580a3d20 0x7f53580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.457+0000 7f5372000640 1 -- 172.21.15.94:0/3430302390 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f53640d0dc0 con 0x7f5374102fb0 2024-04-25T15:12:15.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.457+0000 7f5378c00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f53580a3d20 0x7f53580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.458+0000 7f5378c00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f53580a3d20 0x7f53580a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5360031440 tx=0x7f5360005b50 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:15.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.460+0000 7f5372000640 1 -- 172.21.15.94:0/3430302390 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f536409cdd0 con 0x7f5374102fb0 2024-04-25T15:12:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.656+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f53741a74f0 con 0x7f5374102fb0 2024-04-25T15:12:15.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.657+0000 7f5372000640 1 -- 172.21.15.94:0/3430302390 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f53640a10d0 con 0x7f5374102fb0 2024-04-25T15:12:15.659 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:12:15.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f53580a3d20 msgr2=0x7f53580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f53580a3d20 0x7f53580a61e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5360031440 tx=0x7f5360005b50 comp rx=0 tx=0).stop 2024-04-25T15:12:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374102fb0 msgr2=0x7f53741a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374102fb0 0x7f53741a0460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f536400e9f0 tx=0x7f536400eeb0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 shutdown_connections 2024-04-25T15:12:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f53580a3d20 0x7f53580a61e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f537410c3d0 0x7f537419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f53741038d0 0x7f53741a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 --2- 172.21.15.94:0/3430302390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5374102fb0 0x7f53741a0460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 >> 172.21.15.94:0/3430302390 conn(0x7f53740fc7d0 msgr2=0x7f5374101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 shutdown_connections 2024-04-25T15:12:15.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.660+0000 7f537aa00640 1 -- 172.21.15.94:0/3430302390 wait complete. 2024-04-25T15:12:15.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T15:12:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 -- 172.21.15.94:0/1842689017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c107250 msgr2=0x7f963c10ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 --2- 172.21.15.94:0/1842689017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c107250 0x7f963c10ee20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9634009ca0 tx=0x7f963402a8c0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 -- 172.21.15.94:0/1842689017 shutdown_connections 2024-04-25T15:12:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 --2- 172.21.15.94:0/1842689017 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c107250 0x7f963c10ee20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 --2- 172.21.15.94:0/1842689017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c102b30 0x7f963c106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 --2- 172.21.15.94:0/1842689017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c102180 0x7f963c102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 -- 172.21.15.94:0/1842689017 >> 172.21.15.94:0/1842689017 conn(0x7f963c079350 msgr2=0x7f963c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 -- 172.21.15.94:0/1842689017 shutdown_connections 2024-04-25T15:12:15.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.884+0000 7f9642000640 1 -- 172.21.15.94:0/1842689017 wait complete. 2024-04-25T15:12:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.885+0000 7f9642000640 1 Processor -- start 2024-04-25T15:12:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.885+0000 7f9642000640 1 -- start start 2024-04-25T15:12:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.885+0000 7f9642000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c102180 0x7f963c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.885+0000 7f9642000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c107250 0x7f963c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.885+0000 7f9642000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c19cfe0 0x7f963c19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9642000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f963c112210 con 0x7f963c107250 2024-04-25T15:12:15.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9642000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f963c112090 con 0x7f963c102180 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9642000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f963c112390 con 0x7f963c19cfe0 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c107250 0x7f963c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c107250 0x7f963c198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54268/0 (socket says 172.21.15.94:54268) 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 -- 172.21.15.94:0/1718485785 learned_addr learned my addr 172.21.15.94:0/1718485785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f963b400640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c102180 0x7f963c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f963be00640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c19cfe0 0x7f963c19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 -- 172.21.15.94:0/1718485785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c19cfe0 msgr2=0x7f963c19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c19cfe0 0x7f963c19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 -- 172.21.15.94:0/1718485785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c102180 msgr2=0x7f963c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c102180 0x7f963c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.886+0000 7f9632a00640 1 -- 172.21.15.94:0/1718485785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c19dad0 con 0x7f963c107250 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.887+0000 7f9632a00640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c107250 0x7f963c198cb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f962400e9c0 tx=0x7f962400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.887+0000 7f9638c00640 1 -- 172.21.15.94:0/1718485785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962400cd80 con 0x7f963c107250 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.887+0000 7f9638c00640 1 -- 172.21.15.94:0/1718485785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96240054d0 con 0x7f963c107250 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.887+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c19ddc0 con 0x7f963c107250 2024-04-25T15:12:15.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.887+0000 7f9638c00640 1 -- 172.21.15.94:0/1718485785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9624010430 con 0x7f963c107250 2024-04-25T15:12:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.887+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a55b0 con 0x7f963c107250 2024-04-25T15:12:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.888+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c0691c0 con 0x7f963c107250 2024-04-25T15:12:15.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.890+0000 7f9638c00640 1 -- 172.21.15.94:0/1718485785 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9624004fd0 con 0x7f963c107250 2024-04-25T15:12:15.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.892+0000 7f9638c00640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f96200a3d20 0x7f96200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.892+0000 7f9638c00640 1 -- 172.21.15.94:0/1718485785 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8586+0+0 (secure 0 0 0) 0x7f96240d0c60 con 0x7f963c107250 2024-04-25T15:12:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.893+0000 7f963b400640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f96200a3d20 0x7f96200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:15.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.893+0000 7f963b400640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f96200a3d20 0x7f96200a61e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f962c009bd0 tx=0x7f962c005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:15.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:15.897+0000 7f9638c00640 1 -- 172.21.15.94:0/1718485785 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f962409d2c0 con 0x7f963c107250 2024-04-25T15:12:15.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:15 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1621468782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:12:15.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:15 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/112710616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:12:15.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:15 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3430302390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:12:16.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.150+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f963c102b30 con 0x7f963c107250 2024-04-25T15:12:16.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.183+0000 7f9638c00640 1 -- 172.21.15.94:0/1718485785 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f962409cbc0 con 0x7f963c107250 2024-04-25T15:12:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:12:16.198 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 26, 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "active_gid": 15027, 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "active_name": "x", 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addrs": { 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2411855535 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2411855535 2024-04-25T15:12:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addr": "172.21.15.94:6801/2411855535", 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "active_change": "2024-04-25T15:02:09.415533+0000", 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "standbys": [ 2024-04-25T15:12:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 25135, 2024-04-25T15:12:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "z", 2024-04-25T15:12:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-04-25T15:12:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-04-25T15:12:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-04-25T15:12:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-04-25T15:12:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-04-25T15:12:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-04-25T15:12:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-04-25T15:12:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-04-25T15:12:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T15:12:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-04-25T15:12:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-04-25T15:12:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-04-25T15:12:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-04-25T15:12:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T15:12:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-04-25T15:12:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T15:12:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-04-25T15:12:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-04-25T15:12:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-04-25T15:12:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-04-25T15:12:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T15:12:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-04-25T15:12:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-04-25T15:12:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-04-25T15:12:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-04-25T15:12:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-04-25T15:12:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-04-25T15:12:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T15:12:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T15:12:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-04-25T15:12:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-04-25T15:12:16.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-04-25T15:12:16.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-04-25T15:12:16.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-04-25T15:12:16.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T15:12:16.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T15:12:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.326 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T15:12:16.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.336 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T15:12:16.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.341 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-04-25T15:12:16.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T15:12:16.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.346 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-04-25T15:12:16.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T15:12:16.350 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.353 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-04-25T15:12:16.354 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T15:12:16.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-04-25T15:12:16.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T15:12:16.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-04-25T15:12:16.365 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.365 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.365 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.365 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.366 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.368 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.369 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.370 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.371 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.372 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-04-25T15:12:16.374 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-04-25T15:12:16.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-04-25T15:12:16.377 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.378 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.379 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.380 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T15:12:16.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.384 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T15:12:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T15:12:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.389 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T15:12:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T15:12:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.391 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.392 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.393 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.394 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T15:12:16.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T15:12:16.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:16.401 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T15:12:16.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.403 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.404 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T15:12:16.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T15:12:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T15:12:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.409 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.410 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.411 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T15:12:16.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T15:12:16.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.418 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T15:12:16.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.423 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.424 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.425 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.427 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.430 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.430 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.430 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.430 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.431 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T15:12:16.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.434 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.435 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-04-25T15:12:16.439 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T15:12:16.440 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T15:12:16.445 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.446 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.447 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.449 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T15:12:16.450 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.452 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.452 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.452 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T15:12:16.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T15:12:16.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-04-25T15:12:16.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.461 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.462 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.464 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.468 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.470 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.471 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.473 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:16.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.476 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.479 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.482 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.485 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.488 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.490 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.491 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.494 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.499 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-04-25T15:12:16.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T15:12:16.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.510 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.511 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.514 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.517 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.519 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.521 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:16.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.522 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.522 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-04-25T15:12:16.523 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-04-25T15:12:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-04-25T15:12:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-04-25T15:12:16.524 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.525 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.526 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-04-25T15:12:16.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-04-25T15:12:16.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-04-25T15:12:16.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.530 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.531 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.532 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.533 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.540 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:16.544 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-04-25T15:12:16.545 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.546 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.547 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.549 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.550 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.554 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.555 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.558 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.560 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.564 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.566 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-04-25T15:12:16.567 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.569 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.572 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.574 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.575 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.576 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.580 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T15:12:16.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.582 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.583 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.587 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.588 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.592 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-04-25T15:12:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T15:12:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-04-25T15:12:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-04-25T15:12:16.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-04-25T15:12:16.596 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.597 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.602 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.603 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.606 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.607 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.608 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.610 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.611 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.612 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.613 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.614 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.615 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.616 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.618 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.619 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.621 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.622 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.624 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.625 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.626 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.627 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.628 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.629 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.630 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.631 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.632 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.633 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.634 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.635 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.637 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-04-25T15:12:16.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T15:12:16.639 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.642 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.644 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.645 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.646 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.647 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.648 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.650 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.651 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.652 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.653 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.655 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.656 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.657 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.660 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.661 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.664 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.665 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.666 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-04-25T15:12:16.670 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.672 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.674 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.675 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.676 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.678 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.680 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.681 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.682 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.683 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.684 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-04-25T15:12:16.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-04-25T15:12:16.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.684 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.684 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.685 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.686 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.687 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-04-25T15:12:16.687 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.687 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.688 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.690 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.692 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.693 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.694 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.696 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.697 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.698 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.699 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.701 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-04-25T15:12:16.702 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.703 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-04-25T15:12:16.704 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-04-25T15:12:16.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.704 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.704 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-04-25T15:12:16.704 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.704 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:16.705 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.706 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.707 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.708 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-04-25T15:12:16.708 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.708 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.708 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T15:12:16.708 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.709 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.711 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T15:12:16.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.715 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-04-25T15:12:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-04-25T15:12:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-04-25T15:12:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-04-25T15:12:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T15:12:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-04-25T15:12:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-04-25T15:12:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-04-25T15:12:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-04-25T15:12:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-04-25T15:12:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-04-25T15:12:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-04-25T15:12:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-04-25T15:12:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-04-25T15:12:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-04-25T15:12:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-04-25T15:12:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-04-25T15:12:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-04-25T15:12:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-04-25T15:12:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.815 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.816 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.817 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.818 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.819 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.821 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.822 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.822 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.822 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.822 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.822 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.822 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.822 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.823 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.824 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.825 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.826 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.827 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.828 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-04-25T15:12:16.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.831 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.833 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.834 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.835 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.836 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.838 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.839 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.840 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.841 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.842 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.843 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.844 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.845 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.846 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.848 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.849 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.850 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.851 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.853 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.854 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.854 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.854 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.854 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.854 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.854 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T15:12:16.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.857 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.858 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.859 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-04-25T15:12:16.860 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-04-25T15:12:16.861 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.862 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.864 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.865 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.866 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.869 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.870 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.871 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.871 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.871 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.871 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.871 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.872 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-04-25T15:12:16.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.874 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.875 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.877 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-04-25T15:12:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.880 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.882 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.884 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.885 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.888 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.889 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.890 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.891 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.892 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.893 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.894 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.895 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.896 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.897 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.899 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.900 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.902 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.904 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.905 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-04-25T15:12:16.906 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.907 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.909 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.910 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.911 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.912 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.913 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.914 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.915 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.916 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.917 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-04-25T15:12:16.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.919 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.920 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-04-25T15:12:16.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:16.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 25139, 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "y", 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.924 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:16.925 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.926 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.926 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T15:12:16.926 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.926 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.927 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.928 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.932 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.933 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.934 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-04-25T15:12:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-04-25T15:12:16.937 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.938 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.939 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-04-25T15:12:16.940 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.941 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.941 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.941 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.941 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.941 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.941 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.942 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.944 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.945 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.946 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.947 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.948 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.949 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-04-25T15:12:16.950 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-04-25T15:12:16.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.950 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.950 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-04-25T15:12:16.951 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-04-25T15:12:16.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.952 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.952 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-04-25T15:12:16.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.952 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.952 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.953 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:12:16.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-04-25T15:12:16.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.956 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.957 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-04-25T15:12:16.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T15:12:16.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-04-25T15:12:16.960 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-04-25T15:12:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.962 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.963 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.964 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.965 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.966 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.967 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.968 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.969 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.970 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-04-25T15:12:16.971 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T15:12:16.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.973 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-04-25T15:12:16.974 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-04-25T15:12:16.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.977 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.979 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:16.980 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.981 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:16.982 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.983 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-04-25T15:12:16.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T15:12:16.985 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.988 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-04-25T15:12:16.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T15:12:16.990 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.991 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.994 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-04-25T15:12:16.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-04-25T15:12:16.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:16.998 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:16.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.001 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-04-25T15:12:17.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.004 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-04-25T15:12:17.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T15:12:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.006 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.007 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.009 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.010 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.011 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-04-25T15:12:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-04-25T15:12:17.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-04-25T15:12:17.013 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T15:12:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-04-25T15:12:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-04-25T15:12:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-04-25T15:12:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.017 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-04-25T15:12:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-04-25T15:12:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-04-25T15:12:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-04-25T15:12:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T15:12:17.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.021 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.022 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-04-25T15:12:17.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.025 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.027 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-04-25T15:12:17.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.030 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.031 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.032 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T15:12:17.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T15:12:17.034 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.037 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-04-25T15:12:17.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-04-25T15:12:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-04-25T15:12:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T15:12:17.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-04-25T15:12:17.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.045 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.047 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.049 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.050 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.051 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.052 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.053 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T15:12:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.054 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.056 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.057 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.058 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.059 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.060 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.061 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.062 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.063 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.064 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-04-25T15:12:17.065 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.066 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.067 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.068 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.070 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T15:12:17.071 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-04-25T15:12:17.072 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T15:12:17.073 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.074 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.076 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-04-25T15:12:17.077 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T15:12:17.078 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.081 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-04-25T15:12:17.082 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T15:12:17.083 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.084 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.085 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.087 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.089 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-04-25T15:12:17.090 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T15:12:17.091 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-04-25T15:12:17.092 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.093 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.094 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.098 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.099 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-04-25T15:12:17.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.102 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T15:12:17.103 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.104 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.106 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.107 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.109 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.110 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.111 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.114 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.117 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T15:12:17.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.119 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.121 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.122 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T15:12:17.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.124 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.127 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T15:12:17.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.134 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T15:12:17.134 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T15:12:17.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.134 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:17.134 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T15:12:17.135 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.136 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.137 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.139 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.142 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T15:12:17.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.149 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T15:12:17.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.153 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T15:12:17.155 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.156 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.157 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.159 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T15:12:17.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.161 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.162 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T15:12:17.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.169 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.170 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.171 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T15:12:17.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.174 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.177 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T15:12:17.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.179 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.181 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.182 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.183 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.184 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-04-25T15:12:17.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.187 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.190 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.197 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.198 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.199 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.200 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.201 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.203 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.204 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-04-25T15:12:17.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.210 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.212 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-04-25T15:12:17.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-04-25T15:12:17.215 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.217 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.218 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.221 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.223 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.224 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-04-25T15:12:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T15:12:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.225 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-04-25T15:12:17.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-04-25T15:12:17.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T15:12:17.229 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.230 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.231 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:17.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-04-25T15:12:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T15:12:17.235 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-04-25T15:12:17.236 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.238 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.242 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-04-25T15:12:17.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.246 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.248 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.252 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.254 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-04-25T15:12:17.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.258 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-04-25T15:12:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-04-25T15:12:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T15:12:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:12:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-04-25T15:12:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T15:12:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-04-25T15:12:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-04-25T15:12:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-04-25T15:12:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-04-25T15:12:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.401 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T15:12:17.403 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.404 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.405 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.406 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.407 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.407 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.409 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.410 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.410 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.410 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.410 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.411 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.411 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.411 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.413 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.415 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.417 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-04-25T15:12:17.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-04-25T15:12:17.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.417 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.417 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-04-25T15:12:17.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.418 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.419 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.421 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.423 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.426 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.430 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.431 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.433 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.433 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.433 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-04-25T15:12:17.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-04-25T15:12:17.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.434 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.434 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.434 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.434 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.434 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.434 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.435 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-04-25T15:12:17.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.438 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.438 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.438 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.438 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.438 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.439 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.440 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.441 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.442 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T15:12:17.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.448 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.449 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.450 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.450 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.450 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.450 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.450 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.450 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.450 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.452 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.453 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.454 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.454 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.454 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.454 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.455 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.457 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.457 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.457 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T15:12:17.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.463 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.468 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.472 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.473 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.477 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.478 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.478 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.478 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.478 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.480 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.483 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.484 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.489 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.490 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.490 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.490 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.490 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.490 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.490 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.491 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.494 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-04-25T15:12:17.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.499 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.505 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.505 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.505 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.505 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.505 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.506 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.506 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-04-25T15:12:17.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.509 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.510 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-04-25T15:12:17.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-04-25T15:12:17.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-04-25T15:12:17.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.516 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.517 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.519 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.520 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.521 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-04-25T15:12:17.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.523 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-04-25T15:12:17.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.529 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.532 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-04-25T15:12:17.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-04-25T15:12:17.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.534 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.535 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.536 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.537 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.539 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.544 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.549 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.550 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.552 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.554 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.558 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.566 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.569 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.570 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.576 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-04-25T15:12:17.577 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.578 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.579 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.580 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-04-25T15:12:17.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T15:12:17.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.589 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-04-25T15:12:17.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.594 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-04-25T15:12:17.594 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-04-25T15:12:17.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.594 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.595 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.596 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.597 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.598 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.598 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.601 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.602 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.603 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.607 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.607 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.607 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.607 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.607 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.610 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.611 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.611 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.611 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.611 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.611 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.613 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.614 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.614 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.614 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.614 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.614 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.616 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.617 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.618 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.618 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.618 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.618 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.618 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.618 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T15:12:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.622 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.623 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.624 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.625 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.625 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.625 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.625 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.625 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.625 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.628 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-04-25T15:12:17.632 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-04-25T15:12:17.633 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.634 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.635 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.636 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-04-25T15:12:17.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.640 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.641 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.642 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.643 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.644 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.646 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.648 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.649 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.650 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.651 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.652 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-04-25T15:12:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T15:12:17.655 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.656 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard", 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-04-25T15:12:17.657 INFO:teuthology.orchestra.run.smithi094.stdout: "restful", 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_schedule" 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.658 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.659 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.660 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.661 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.663 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.664 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.666 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.668 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-04-25T15:12:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-04-25T15:12:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-04-25T15:12:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.670 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-04-25T15:12:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-04-25T15:12:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-04-25T15:12:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-04-25T15:12:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-04-25T15:12:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-04-25T15:12:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-04-25T15:12:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.674 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.675 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.676 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.677 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-04-25T15:12:17.678 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.679 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.680 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-04-25T15:12:17.681 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-04-25T15:12:17.682 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.683 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.684 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-04-25T15:12:17.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T15:12:17.686 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.689 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.690 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T15:12:17.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.692 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-04-25T15:12:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-04-25T15:12:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-04-25T15:12:17.694 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T15:12:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T15:12:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.695 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:17.696 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.697 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.698 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.699 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.703 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.704 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.705 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.707 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.710 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T15:12:17.711 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.713 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-04-25T15:12:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-04-25T15:12:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:17.714 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-04-25T15:12:17.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T15:12:17.716 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.717 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T15:12:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-04-25T15:12:17.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.720 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T15:12:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-04-25T15:12:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-04-25T15:12:17.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T15:12:17.722 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.723 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.724 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.725 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T15:12:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.726 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-04-25T15:12:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T15:12:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-04-25T15:12:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.727 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-04-25T15:12:17.728 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-04-25T15:12:17.729 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.730 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.731 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.733 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.734 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-04-25T15:12:17.735 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-04-25T15:12:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T15:12:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.736 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-04-25T15:12:17.737 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T15:12:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.738 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.739 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T15:12:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-04-25T15:12:17.740 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-04-25T15:12:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-04-25T15:12:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.741 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-04-25T15:12:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-04-25T15:12:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.743 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T15:12:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-04-25T15:12:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T15:12:17.745 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-04-25T15:12:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-04-25T15:12:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-04-25T15:12:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T15:12:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_samba": { 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_samba", 2024-04-25T15:12:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Samba/SMB container image", 2024-04-25T15:12:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-04-25T15:12:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-04-25T15:12:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-04-25T15:12:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "grafana_dashboards_path": { 2024-04-25T15:12:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "grafana_dashboards_path", 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T15:12:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T15:12:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T15:12:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T15:12:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T15:12:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-04-25T15:12:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-04-25T15:12:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-04-25T15:12:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-04-25T15:12:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-04-25T15:12:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T15:12:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-04-25T15:12:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-04-25T15:12:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-04-25T15:12:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-04-25T15:12:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T15:12:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T15:12:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-04-25T15:12:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-04-25T15:12:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T15:12:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-04-25T15:12:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-04-25T15:12:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T15:12:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T15:12:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T15:12:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T15:12:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-04-25T15:12:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T15:12:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T15:12:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T15:12:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T15:12:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T15:12:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T15:12:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T15:12:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T15:12:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T15:12:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T15:12:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T15:12:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T15:12:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "[]", 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T15:12:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "{}", 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T15:12:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T15:12:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T15:12:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T15:12:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T15:12:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T15:12:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T15:12:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T15:12:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T15:12:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T15:12:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T15:12:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T15:12:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T15:12:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T15:12:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T15:12:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T15:12:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T15:12:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T15:12:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-04-25T15:12:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T15:12:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T15:12:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T15:12:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T15:12:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T15:12:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T15:12:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T15:12:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T15:12:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T15:12:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T15:12:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T15:12:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T15:12:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T15:12:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-04-25T15:12:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-04-25T15:12:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-04-25T15:12:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T15:12:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T15:12:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-04-25T15:12:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T15:12:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.969 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.970 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.971 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.973 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.973 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-04-25T15:12:17.973 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-04-25T15:12:17.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.974 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.975 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-04-25T15:12:17.976 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-04-25T15:12:17.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.977 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.977 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.977 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-04-25T15:12:17.977 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.977 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.977 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.978 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-04-25T15:12:17.978 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.978 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.978 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-04-25T15:12:17.978 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.979 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.981 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.982 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.983 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.985 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T15:12:17.985 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.985 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.985 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:17.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:17.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-04-25T15:12:17.986 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:17.986 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:17.986 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:17.986 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:17.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.987 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.989 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:17.989 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:17.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.989 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.990 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:17.992 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:17.993 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.995 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-04-25T15:12:17.996 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.996 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.996 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.996 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.996 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.996 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-04-25T15:12:17.997 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:17.998 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:17.999 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-04-25T15:12:18.000 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-04-25T15:12:18.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-04-25T15:12:18.002 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.006 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.006 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:18.006 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:18.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:18.006 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.007 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-04-25T15:12:18.007 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-04-25T15:12:18.007 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.007 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.007 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T15:12:18.007 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.008 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.009 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.010 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.010 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.010 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.010 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.010 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.011 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.012 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.013 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.013 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.013 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.013 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.014 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.015 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.016 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.018 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-04-25T15:12:18.019 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.020 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.021 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-04-25T15:12:18.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.024 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-04-25T15:12:18.024 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-04-25T15:12:18.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.024 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.025 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-04-25T15:12:18.026 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.028 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.029 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.030 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.030 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.030 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.030 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.030 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.030 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.031 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.032 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.033 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.034 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.034 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.034 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.034 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.034 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.034 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.037 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.038 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.040 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.042 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.042 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.043 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.044 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.044 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.044 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.044 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.044 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.045 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.046 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.049 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.051 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.052 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.054 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.054 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.054 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.054 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.054 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.054 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.054 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.056 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.058 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.059 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-04-25T15:12:18.060 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.061 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.062 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.063 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.066 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.067 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.069 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-04-25T15:12:18.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-04-25T15:12:18.073 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.074 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.075 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.076 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.078 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.079 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.081 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.082 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.083 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.084 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.088 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.090 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.091 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.092 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.093 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.096 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.097 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.099 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.100 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.101 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.107 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.108 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.109 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.110 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.112 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-04-25T15:12:18.116 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.119 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.121 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.123 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.124 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.127 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.128 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.130 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.131 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.132 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.135 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.135 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.135 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.136 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-04-25T15:12:18.136 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.136 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.137 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.138 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.138 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.138 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.138 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.138 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.138 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.138 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.139 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.139 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.139 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.139 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.140 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.142 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.145 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:18.146 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:18.146 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:18.146 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.146 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.146 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:18.146 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.146 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:18.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.149 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.150 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.152 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.153 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.154 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.155 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.156 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.157 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.158 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.160 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-04-25T15:12:18.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-04-25T15:12:18.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.164 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.165 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.166 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-04-25T15:12:18.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.170 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.173 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.174 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.176 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.177 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.179 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.179 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T15:12:18.179 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T15:12:18.179 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.179 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.179 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.181 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.182 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.182 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.182 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.182 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.182 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T15:12:18.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.186 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-04-25T15:12:18.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-04-25T15:12:18.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.186 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.186 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.187 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-04-25T15:12:18.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.190 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.191 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.193 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.194 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.194 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.194 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.194 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.197 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.198 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.198 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.198 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.200 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.200 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.200 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T15:12:18.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.203 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-04-25T15:12:18.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-04-25T15:12:18.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.207 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.207 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.207 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-04-25T15:12:18.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.210 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.211 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.212 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.214 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.215 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.219 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.222 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.223 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.225 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.226 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.227 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.230 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.231 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.233 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.235 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.237 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.239 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.240 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.241 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.244 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.245 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.246 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.246 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.246 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.246 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.246 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.246 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.250 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.254 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.258 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-04-25T15:12:18.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-04-25T15:12:18.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.259 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.260 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.261 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.265 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.266 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-04-25T15:12:18.267 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-04-25T15:12:18.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.267 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.267 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-04-25T15:12:18.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T15:12:18.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.277 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-04-25T15:12:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-04-25T15:12:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-04-25T15:12:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-04-25T15:12:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-04-25T15:12:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-04-25T15:12:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-04-25T15:12:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-04-25T15:12:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-04-25T15:12:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-04-25T15:12:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-04-25T15:12:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-04-25T15:12:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_no_wait": { 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-04-25T15:12:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-04-25T15:12:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-04-25T15:12:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-04-25T15:12:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-04-25T15:12:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-04-25T15:12:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-04-25T15:12:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T15:12:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: "services": { 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard": "https://172.21.15.94:8443/" 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: "always_on_modules": { 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: "octopus": [ 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T15:12:18.402 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T15:12:18.403 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "pacific": [ 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T15:12:18.404 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: "quincy": [ 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T15:12:18.405 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T15:12:18.406 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T15:12:18.406 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T15:12:18.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T15:12:18.406 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T15:12:18.406 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T15:12:18.406 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T15:12:18.406 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "reef": [ 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T15:12:18.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: "squid": [ 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-04-25T15:12:18.408 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: "last_failure_osd_epoch": 89, 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: "active_clients": [ 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T15:12:18.410 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1881735427 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "libcephsqlite", 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T15:12:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 263111784 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 4286782282 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T15:12:18.414 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T15:12:18.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T15:12:18.414 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T15:12:18.414 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1195198346 2024-04-25T15:12:18.414 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T15:12:18.415 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3496315578 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-04-25T15:12:18.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-04-25T15:12:18.417 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-04-25T15:12:18.417 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1468796874 2024-04-25T15:12:18.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.417 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:18.417 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-04-25T15:12:18.417 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T15:12:18.419 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:15 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1621468782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:12:18.419 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:15 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/112710616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:12:18.419 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:15 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3430302390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:12:18.419 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:16 smithi137 ceph-mon[32439]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-25T15:12:18.420 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:16 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1718485785' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T15:12:18.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:15 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1621468782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:12:18.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:15 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/112710616' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:12:18.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:15 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3430302390' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:12:18.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:16 smithi138 ceph-mon[32875]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-25T15:12:18.420 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:16 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1718485785' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T15:12:18.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f96200a3d20 msgr2=0x7f96200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:18.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f96200a3d20 0x7f96200a61e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f962c009bd0 tx=0x7f962c005f90 comp rx=0 tx=0).stop 2024-04-25T15:12:18.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c107250 msgr2=0x7f963c198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c107250 0x7f963c198cb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f962400e9c0 tx=0x7f962400ee80 comp rx=0 tx=0).stop 2024-04-25T15:12:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 shutdown_connections 2024-04-25T15:12:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f96200a3d20 0x7f96200a61e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f963c19cfe0 0x7f963c19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f963c107250 0x7f963c198cb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 --2- 172.21.15.94:0/1718485785 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f963c102180 0x7f963c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:18.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 >> 172.21.15.94:0/1718485785 conn(0x7f963c079350 msgr2=0x7f963c1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:18.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 shutdown_connections 2024-04-25T15:12:18.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:16.198+0000 7f9642000640 1 -- 172.21.15.94:0/1718485785 wait complete. 2024-04-25T15:12:18.432 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T15:12:18.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T15:12:18.433 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:16 smithi094 ceph-mon[27862]: pgmap v313: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 5 op/s 2024-04-25T15:12:18.433 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:16 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1718485785' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T15:12:18.511 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:18 smithi094 ceph-mon[27862]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T15:12:18.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:18 smithi138 ceph-mon[32875]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T15:12:18.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:18 smithi137 ceph-mon[32439]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T15:12:18.894 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:12:19.366 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:19 smithi094 ceph-mon[27862]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T15:12:19.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:19 smithi138 ceph-mon[32875]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T15:12:19.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:19 smithi137 ceph-mon[32439]: pgmap v315: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T15:12:21.157 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T15:12:21.158 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:21.158 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:21.158 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:21.158 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:21.158 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:21.158 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:21.158 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.004003279, 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000666292, 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:21.159 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:21.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 6, 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 159, 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 3264, 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 184064, 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:21.161 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.034025014, 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.022523126, 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.006198409, 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 157, 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.020270920, 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000129114 2024-04-25T15:12:21.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 184064 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.163 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 216, 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 205, 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1588016, 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 125984, 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.101369625, 2024-04-25T15:12:21.164 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.032112865, 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.082938181, 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.009404330, 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 204, 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.020724981, 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000101593 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:21.165 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1588016, 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 125984 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T15:12:21.166 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:21.167 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:21.168 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:21.169 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:21.170 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148977, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-04-25T15:12:21.171 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:21.172 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:21.173 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:21.174 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 218, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1557580, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 218, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 218, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1557580, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:21.175 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:21.176 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:21.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:21.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:21.176 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T15:12:21.588 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:21 smithi094 ceph-mon[27862]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 114 B/s wr, 4 op/s 2024-04-25T15:12:21.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:21 smithi138 ceph-mon[32875]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 114 B/s wr, 4 op/s 2024-04-25T15:12:21.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:21 smithi137 ceph-mon[32439]: pgmap v316: 97 pgs: 97 active+clean; 1.3 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 114 B/s wr, 4 op/s 2024-04-25T15:12:21.999 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T15:12:21.999 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T15:12:21.999 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T15:12:21.999 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T15:12:21.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T15:12:22.453 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:12:23.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:23 smithi138 ceph-mon[32875]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T15:12:23.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:23 smithi094 ceph-mon[27862]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T15:12:23.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:23 smithi137 ceph-mon[32439]: pgmap v317: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T15:12:24.524 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T15:12:24.524 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:24.524 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-04-25T15:12:24.524 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 4, 2024-04-25T15:12:24.524 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:12:24.524 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 448, 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.004465294, 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000077201, 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.001446236, 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:24.525 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.001605963, 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000535321 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:24.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 2, 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 161, 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:12:24.527 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 184320, 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.034076709, 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.024995008, 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.003833192, 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:24.528 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 158, 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.023904754, 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000151295 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:24.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 184320 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 223, 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 212, 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1589920, 2024-04-25T15:12:24.530 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 130464, 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.099422050, 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.032860857, 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.078709419, 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.010234943, 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:24.531 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 211, 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.021470438, 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000101755 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.532 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1589920, 2024-04-25T15:12:24.533 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 130464 2024-04-25T15:12:24.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.533 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T15:12:24.533 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-04-25T15:12:24.533 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T15:12:24.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.533 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:24.534 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:24.535 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:24.536 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T15:12:24.537 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:24.538 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:24.538 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:24.538 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:24.538 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 150008, 2024-04-25T15:12:24.538 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 31, 2024-04-25T15:12:24.538 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:24.538 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-04-25T15:12:24.539 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T15:12:24.540 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 225, 2024-04-25T15:12:24.541 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1558490, 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 225, 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 225, 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1558490, 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:24.542 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:24.543 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:24.543 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:24.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:24.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:24.543 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T15:12:25.326 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T15:12:25.326 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T15:12:25.326 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T15:12:25.326 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T15:12:25.326 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T15:12:25.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:25 smithi138 ceph-mon[32875]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T15:12:25.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:25 smithi094 ceph-mon[27862]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T15:12:25.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:25 smithi137 ceph-mon[32439]: pgmap v318: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T15:12:25.764 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:12:27.324 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:27 smithi094 ceph-mon[27862]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:12:27.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:27 smithi138 ceph-mon[32875]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:12:27.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:27 smithi137 ceph-mon[32439]: pgmap v319: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:12:27.917 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T15:12:27.917 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:27.917 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 1562, 2024-04-25T15:12:27.917 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 1564, 2024-04-25T15:12:27.917 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 555639, 2024-04-25T15:12:27.917 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 496726, 2024-04-25T15:12:27.918 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-04-25T15:12:27.918 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-04-25T15:12:27.918 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.318539191, 2024-04-25T15:12:27.918 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.115690901, 2024-04-25T15:12:27.918 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.081184737, 2024-04-25T15:12:27.918 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.085425190, 2024-04-25T15:12:27.918 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 1562, 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.129927877, 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000083180 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 10, 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 165, 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 4920, 2024-04-25T15:12:27.920 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 446714, 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 7, 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 5, 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.036703548, 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.024979212, 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.005970756, 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000099682, 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:27.921 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 163, 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.029936220, 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000183657 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 445376 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 974, 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 962, 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1790013, 2024-04-25T15:12:27.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 365199, 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 3, 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 2, 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.252455941, 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.078773742, 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.120772294, 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.061506301, 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:27.924 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 961, 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.089397100, 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000093025 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1608640, 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 130832 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 1, 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.926 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-MDSRank": { 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3839, 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.206096983, 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000053685 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.927 INFO:teuthology.orchestra.run.smithi094.stdout: "finisher-PurgeQueue": { 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: "queue_len": 0, 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: "complete_latency": { 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: "mds": { 2024-04-25T15:12:27.928 INFO:teuthology.orchestra.run.smithi094.stdout: "request": 0, 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: "reply": 0, 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: "reply_latency": { 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_reply": 0, 2024-04-25T15:12:27.929 INFO:teuthology.orchestra.run.smithi094.stdout: "forward": 0, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_complete": 0, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_fetch_keys": 0, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_commit": 0, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_split": 0, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_merge": 0, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes": 12, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_top": 0, 2024-04-25T15:12:27.930 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_bottom": 12, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pin_tail": 0, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_pinned": 4, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_expired": 0, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "inodes_with_caps": 0, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "caps": 0, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "subtrees": 2, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse": 0, 2024-04-25T15:12:27.931 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_hit": 0, 2024-04-25T15:12:27.932 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_forward": 0, 2024-04-25T15:12:27.932 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_discover": 0, 2024-04-25T15:12:27.932 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_dir_fetch": 0, 2024-04-25T15:12:27.932 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_remote_ino": 0, 2024-04-25T15:12:27.932 INFO:teuthology.orchestra.run.smithi094.stdout: "traverse_lock": 0, 2024-04-25T15:12:27.932 INFO:teuthology.orchestra.run.smithi094.stdout: "load_cent": 0, 2024-04-25T15:12:27.932 INFO:teuthology.orchestra.run.smithi094.stdout: "q": 0, 2024-04-25T15:12:27.933 INFO:teuthology.orchestra.run.smithi094.stdout: "exported": 0, 2024-04-25T15:12:27.933 INFO:teuthology.orchestra.run.smithi094.stdout: "exported_inodes": 0, 2024-04-25T15:12:27.933 INFO:teuthology.orchestra.run.smithi094.stdout: "imported": 0, 2024-04-25T15:12:27.933 INFO:teuthology.orchestra.run.smithi094.stdout: "imported_inodes": 0, 2024-04-25T15:12:27.933 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_dir_fetch": 0, 2024-04-25T15:12:27.933 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_backtrace_fetch": 0, 2024-04-25T15:12:27.933 INFO:teuthology.orchestra.run.smithi094.stdout: "openino_peer_discover": 0, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rfiles": 4, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rbytes": 234, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "root_rsnaps": 6, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_set_tag": 0, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_inotable_repaired": 0, 2024-04-25T15:12:27.934 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_inodes": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_file_inodes": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_inode_file_caps": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_grant": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T15:12:27.935 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps": 0, 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_caps_dirty": 0, 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_cap_release": 0, 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: "process_request_cap_release": 0 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache": { 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update": 0, 2024-04-25T15:12:27.936 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_update_receipt": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_try_discover": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_send_discover": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "dir_handle_discover": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_delayed": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "num_strays_enqueuing": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_created": 0, 2024-04-25T15:12:27.937 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_enqueued": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_reintegrated": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "strays_migrated": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_processing": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_enqueued": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "num_recovering_prioritized": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_started": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_completed": 0, 2024-04-25T15:12:27.938 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_path": 0, 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_quiesce_inode": 0, 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_exportdir": 0, 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_flush": 0, 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragmentdir": 0, 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_fragstats": 0, 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: "ireq_inodestats": 0 2024-04-25T15:12:27.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log": { 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "evlrg": 0, 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "evadd": 0, 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "evex": 0, 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "evtrm": 0, 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "ev": 101, 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "evexg": 0, 2024-04-25T15:12:27.940 INFO:teuthology.orchestra.run.smithi094.stdout: "evexd": 0, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "segadd": 0, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "segex": 0, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "segtrm": 0, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "seg": 2, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "segmjr": 3, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "segexg": 0, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "segexd": 0, 2024-04-25T15:12:27.941 INFO:teuthology.orchestra.run.smithi094.stdout: "expos": 4261392, 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: "wrpos": 4284828, 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: "rdpos": 4284716, 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: "jlat": { 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.942 INFO:teuthology.orchestra.run.smithi094.stdout: "replayed": 101 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mem": { 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: "ino": 13, 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: "ino+": 13, 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: "ino-": 0, 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: "dir": 8, 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: "dir+": 8, 2024-04-25T15:12:27.943 INFO:teuthology.orchestra.run.smithi094.stdout: "dir-": 0, 2024-04-25T15:12:27.944 INFO:teuthology.orchestra.run.smithi094.stdout: "dn": 12, 2024-04-25T15:12:27.944 INFO:teuthology.orchestra.run.smithi094.stdout: "dn+": 12, 2024-04-25T15:12:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: "dn-": 0, 2024-04-25T15:12:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: "cap": 0, 2024-04-25T15:12:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: "cap+": 0, 2024-04-25T15:12:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: "cap-": 0, 2024-04-25T15:12:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: "rss": 49792, 2024-04-25T15:12:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: "heap": 215416 2024-04-25T15:12:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server": { 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_client_request": 0, 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "dispatch_server_request": 0, 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_request": 0, 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_client_session": 0, 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "handle_peer_request": 0, 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "req_create_latency": { 2024-04-25T15:12:27.946 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getattr_latency": { 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.947 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getfilelock_latency": { 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: "req_link_latency": { 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.948 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookup_latency": { 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookuphash_latency": { 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.949 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupino_latency": { 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.950 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupname_latency": { 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupparent_latency": { 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.951 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.952 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lookupsnap_latency": { 2024-04-25T15:12:27.952 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.952 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.952 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.953 INFO:teuthology.orchestra.run.smithi094.stdout: "req_lssnap_latency": { 2024-04-25T15:12:27.953 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.953 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.953 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.953 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mkdir_latency": { 2024-04-25T15:12:27.953 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mknod_latency": { 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "req_mksnap_latency": { 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "req_open_latency": { 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.956 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "req_readdir_latency": { 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rename_latency": { 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.957 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "req_renamesnap_latency": { 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.958 INFO:teuthology.orchestra.run.smithi094.stdout: "req_snapdiff_latency": { 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmdir_latency": { 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmsnap_latency": { 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "req_rmxattr_latency": { 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.960 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setattr_latency": { 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setdirlayout_latency": { 2024-04-25T15:12:27.961 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setfilelock_latency": { 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setlayout_latency": { 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "req_setxattr_latency": { 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.963 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "req_symlink_latency": { 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "req_unlink_latency": { 2024-04-25T15:12:27.964 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_revoke_eviction": 0, 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "cap_acquisition_throttle": 0, 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "req_getvxattr_latency": { 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.965 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessions": { 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "session_count": 2, 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "session_add": 5, 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "session_remove": 3, 2024-04-25T15:12:27.966 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_open": 4, 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "sessions_stale": 0, 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "total_load": 0, 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "average_load": 0, 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "avg_session_uptime": 3165, 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "mdthresh_evicted": 0 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T15:12:27.967 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:27.968 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:27.969 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:27.970 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:27.971 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 190424, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 52, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:27.972 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 8128, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 169, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:27.973 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 47746, 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 220, 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.974 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter": { 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "op_active": 0, 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "op_laggy": 0, 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send": 2303, 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "op_send_bytes": 26, 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "op_resend": 4, 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "op_reply": 2303, 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "op_latency": { 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2303, 2024-04-25T15:12:27.975 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 1.784970836, 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000775063 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "op_inflight": 0, 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "oplen_avg": { 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 2303, 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 2304 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "op": 2303, 2024-04-25T15:12:27.976 INFO:teuthology.orchestra.run.smithi094.stdout: "op_r": 2303, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "op_w": 0, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "op_rmw": 0, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "op_pg": 0, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_stat": 1493, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_create": 0, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_read": 807, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_write": 0, 2024-04-25T15:12:27.977 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writefull": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_writesame": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_append": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_zero": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_truncate": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_delete": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_mapext": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_sparse_read": 0, 2024-04-25T15:12:27.978 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_clonerange": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_getxattr": 2, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_setxattr": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_cmpxattr": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_rmxattr": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_resetxattrs": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_call": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_watch": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_notify": 0, 2024-04-25T15:12:27.979 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_pgls_filter": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "osdop_other": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_active": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_send": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_resend": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "linger_ping": 0, 2024-04-25T15:12:27.980 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_active": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_send": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "poolop_resend": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_active": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_send": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "poolstat_resend": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_active": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_send": 0, 2024-04-25T15:12:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: "statfs_resend": 0, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "command_active": 0, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "command_send": 0, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "command_resend": 0, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "map_epoch": 97, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "map_full": 0, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "map_inc": 11, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_sessions": 5, 2024-04-25T15:12:27.982 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_open": 5, 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_session_close": 0, 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_laggy": 0, 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_wr": 0, 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_rd": 2, 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_del": 0 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "oft": { 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_objs": 0, 2024-04-25T15:12:27.983 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_kv_pairs": 0, 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_updates": 0, 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "omap_total_removes": 0 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "osd-slow-ops": { 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "slow_ops_count": 0 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "purge_queue": { 2024-04-25T15:12:27.984 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops": 0, 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing": 0, 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executing_high_water": 0, 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed_ops": 0, 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_executed": 0, 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "pq_item_in_journal": 0 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:27.985 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 2542, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 2106728, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 2542, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T15:12:27.986 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2542, 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2106728, 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.987 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_bytes": { 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T15:12:27.988 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2303, 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 90534, 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2303, 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 90534, 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-objecter_ops": { 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 1024, 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:27.990 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 2303, 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 2303, 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 2303, 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 2303, 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.991 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle": { 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T15:12:27.992 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.993 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-write_buf_throttle-0x5649a21bc780": { 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 3758096384, 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 0, 2024-04-25T15:12:27.994 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:27.995 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:27.996 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T15:12:28.858 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-04-25T15:12:28.858 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-25T15:12:28.858 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T15:12:28.859 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T15:12:28.859 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T15:12:29.312 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T15:12:29.630 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:29 smithi094 ceph-mon[27862]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 977 B/s rd, 1 op/s 2024-04-25T15:12:29.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:29 smithi138 ceph-mon[32875]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 977 B/s rd, 1 op/s 2024-04-25T15:12:29.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:29 smithi137 ceph-mon[32439]: pgmap v320: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 977 B/s rd, 1 op/s 2024-04-25T15:12:31.431 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-04-25T15:12:31.431 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:31.431 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:31.431 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:31.431 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.003644815, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000735215, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:31.432 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:31.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 225, 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 214, 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1590464, 2024-04-25T15:12:31.434 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 131744, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.097988740, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.033620449, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.075480928, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.009819878, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 213, 2024-04-25T15:12:31.435 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.022365137, 2024-04-25T15:12:31.436 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000105000 2024-04-25T15:12:31.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.436 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:31.436 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:31.436 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:31.436 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1590464, 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 131744 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 6, 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 166, 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 3264, 2024-04-25T15:12:31.437 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 185296, 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.036976657, 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.025778904, 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.005671533, 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:31.438 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 164, 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.027427436, 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000167240 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.439 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 185296 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:31.440 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:31.441 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:31.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:31.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:31.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:31.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:31.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:31.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:31.442 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:31.443 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:31.444 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 150008, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 31, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:31.445 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-04-25T15:12:31.446 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:31.447 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 227, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1558750, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 227, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-04-25T15:12:31.448 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 227, 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1558750, 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-04-25T15:12:31.449 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-04-25T15:12:31.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:31 smithi138 ceph-mon[32875]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:31.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:31 smithi094 ceph-mon[27862]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:31.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:31 smithi137 ceph-mon[32439]: pgmap v321: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:32.302 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T15:12:32.302 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T15:12:32.303 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T15:12:32.303 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T15:12:32.303 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T15:12:32.554 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:12:33.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:33 smithi138 ceph-mon[32875]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:33.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:33 smithi094 ceph-mon[27862]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:33.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:33 smithi137 ceph-mon[32439]: pgmap v322: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:34.313 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:34 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:12:34.315 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T15:12:34.315 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:34.315 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1179, 2024-04-25T15:12:34.315 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 621, 2024-04-25T15:12:34.315 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1977172, 2024-04-25T15:12:34.315 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 327565, 2024-04-25T15:12:34.315 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 11, 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 5, 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.151707244, 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.037814882, 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.077600533, 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.045062597, 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 620, 2024-04-25T15:12:34.316 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.053056727, 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000085575 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 813, 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000162698, 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000200 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.317 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1923520, 2024-04-25T15:12:34.318 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 225232 2024-04-25T15:12:34.318 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.318 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:34.318 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 389, 2024-04-25T15:12:34.318 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 247, 2024-04-25T15:12:34.318 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 136832, 2024-04-25T15:12:34.319 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 287564, 2024-04-25T15:12:34.319 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 11, 2024-04-25T15:12:34.319 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-04-25T15:12:34.320 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.031797793, 2024-04-25T15:12:34.320 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.006748483, 2024-04-25T15:12:34.320 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.011259847, 2024-04-25T15:12:34.320 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.012253062, 2024-04-25T15:12:34.320 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:34.320 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 243, 2024-04-25T15:12:34.321 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.191212823, 2024-04-25T15:12:34.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000786884 2024-04-25T15:12:34.321 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.321 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:34.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 206, 2024-04-25T15:12:34.321 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000039807, 2024-04-25T15:12:34.321 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000193 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 86608, 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 219280 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 1036, 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 507, 2024-04-25T15:12:34.322 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 387377, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 591817, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 13, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 6, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.139717878, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.033469498, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.046052077, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.055919042, 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:34.323 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 506, 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.288418017, 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000569996 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 852, 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000153642, 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000180 2024-04-25T15:12:34.324 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 52336, 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 344464 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 1, 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-MDSRank": { 2024-04-25T15:12:34.325 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T15:12:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T15:12:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 248, 2024-04-25T15:12:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.035078814, 2024-04-25T15:12:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000141446 2024-04-25T15:12:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.326 INFO:teuthology.orchestra.run.smithi137.stdout: "finisher-PurgeQueue": { 2024-04-25T15:12:34.327 INFO:teuthology.orchestra.run.smithi137.stdout: "queue_len": 0, 2024-04-25T15:12:34.327 INFO:teuthology.orchestra.run.smithi137.stdout: "complete_latency": { 2024-04-25T15:12:34.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-04-25T15:12:34.327 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000005898, 2024-04-25T15:12:34.327 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000002949 2024-04-25T15:12:34.327 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.327 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "mds": { 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "request": 152, 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "reply": 152, 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "reply_latency": { 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 152, 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.118830196, 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000781777 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.328 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_reply": 0, 2024-04-25T15:12:34.329 INFO:teuthology.orchestra.run.smithi137.stdout: "forward": 0, 2024-04-25T15:12:34.329 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_complete": 0, 2024-04-25T15:12:34.329 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_fetch_keys": 0, 2024-04-25T15:12:34.329 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_commit": 19, 2024-04-25T15:12:34.329 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_split": 0, 2024-04-25T15:12:34.329 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_merge": 0, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes": 25, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_top": 2, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_bottom": 23, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pin_tail": 0, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_pinned": 15, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_expired": 0, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "inodes_with_caps": 2, 2024-04-25T15:12:34.330 INFO:teuthology.orchestra.run.smithi137.stdout: "caps": 2, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "subtrees": 2, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse": 193, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_hit": 185, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_forward": 0, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_discover": 0, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_dir_fetch": 0, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_remote_ino": 0, 2024-04-25T15:12:34.331 INFO:teuthology.orchestra.run.smithi137.stdout: "traverse_lock": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "load_cent": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "q": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "exported": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "exported_inodes": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "imported": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "imported_inodes": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_dir_fetch": 0, 2024-04-25T15:12:34.332 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_backtrace_fetch": 0, 2024-04-25T15:12:34.333 INFO:teuthology.orchestra.run.smithi137.stdout: "openino_peer_discover": 0, 2024-04-25T15:12:34.333 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rfiles": 4, 2024-04-25T15:12:34.333 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rbytes": 234, 2024-04-25T15:12:34.333 INFO:teuthology.orchestra.run.smithi137.stdout: "root_rsnaps": 6, 2024-04-25T15:12:34.333 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T15:12:34.333 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_set_tag": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_inotable_repaired": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_inodes": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "scrub_file_inodes": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_inode_file_caps": 0, 2024-04-25T15:12:34.334 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_grant": 5, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps": 14, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_caps_dirty": 2, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_cap_release": 0, 2024-04-25T15:12:34.335 INFO:teuthology.orchestra.run.smithi137.stdout: "process_request_cap_release": 17 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_cache": { 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update": 0, 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_update_receipt": 0, 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_try_discover": 0, 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_send_discover": 0, 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: "dir_handle_discover": 0, 2024-04-25T15:12:34.336 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_delayed": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "num_strays_enqueuing": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_created": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_enqueued": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_reintegrated": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "strays_migrated": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_processing": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_enqueued": 0, 2024-04-25T15:12:34.337 INFO:teuthology.orchestra.run.smithi137.stdout: "num_recovering_prioritized": 0, 2024-04-25T15:12:34.338 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_started": 0, 2024-04-25T15:12:34.338 INFO:teuthology.orchestra.run.smithi137.stdout: "recovery_completed": 0, 2024-04-25T15:12:34.338 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_path": 0, 2024-04-25T15:12:34.338 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_quiesce_inode": 0, 2024-04-25T15:12:34.338 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T15:12:34.338 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_exportdir": 0, 2024-04-25T15:12:34.338 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_flush": 0, 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragmentdir": 0, 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_fragstats": 0, 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "ireq_inodestats": 0 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_log": { 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "evlrg": 0, 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "evadd": 101, 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "evex": 37, 2024-04-25T15:12:34.339 INFO:teuthology.orchestra.run.smithi137.stdout: "evtrm": 37, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "ev": 64, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "evexg": 0, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "evexd": 36, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "segadd": 3, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "segex": 2, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "segtrm": 2, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "seg": 1, 2024-04-25T15:12:34.340 INFO:teuthology.orchestra.run.smithi137.stdout: "segmjr": 1, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "segexg": 0, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "segexd": 1, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "expos": 4261392, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "wrpos": 4284828, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "rdpos": 0, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "jlat": { 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 81, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.150138774, 2024-04-25T15:12:34.341 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001853565 2024-04-25T15:12:34.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.342 INFO:teuthology.orchestra.run.smithi137.stdout: "replayed": 0 2024-04-25T15:12:34.342 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.342 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_mem": { 2024-04-25T15:12:34.342 INFO:teuthology.orchestra.run.smithi137.stdout: "ino": 23, 2024-04-25T15:12:34.342 INFO:teuthology.orchestra.run.smithi137.stdout: "ino+": 23, 2024-04-25T15:12:34.342 INFO:teuthology.orchestra.run.smithi137.stdout: "ino-": 0, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "dir": 18, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "dir+": 18, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "dir-": 0, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "dn": 25, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "dn+": 25, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "dn-": 0, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "cap": 2, 2024-04-25T15:12:34.343 INFO:teuthology.orchestra.run.smithi137.stdout: "cap+": 17, 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: "cap-": 15, 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: "rss": 54384, 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: "heap": 231800 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_server": { 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_client_request": 174, 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: "dispatch_server_request": 0, 2024-04-25T15:12:34.344 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_request": 152, 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_client_session": 31, 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "handle_peer_request": 0, 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "req_create_latency": { 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.003397139, 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000849284 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getattr_latency": { 2024-04-25T15:12:34.345 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 103, 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.022877635, 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000222112 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getfilelock_latency": { 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.346 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: "req_link_latency": { 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookup_latency": { 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2024-04-25T15:12:34.347 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000976368, 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000244092 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookuphash_latency": { 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.348 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupino_latency": { 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupname_latency": { 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.349 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupparent_latency": { 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lookupsnap_latency": { 2024-04-25T15:12:34.350 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: "req_lssnap_latency": { 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 10, 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.002157079, 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000215707 2024-04-25T15:12:34.351 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.352 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mkdir_latency": { 2024-04-25T15:12:34.352 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 6, 2024-04-25T15:12:34.353 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.006739216, 2024-04-25T15:12:34.353 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001123202 2024-04-25T15:12:34.353 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.353 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mknod_latency": { 2024-04-25T15:12:34.353 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.353 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.353 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: "req_mksnap_latency": { 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 10, 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.042713355, 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.004271335 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: "req_open_latency": { 2024-04-25T15:12:34.354 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000372325, 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000372325 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: "req_readdir_latency": { 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.355 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rename_latency": { 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 2, 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.002264812, 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001132406 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: "req_renamesnap_latency": { 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.356 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: "req_snapdiff_latency": { 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmdir_latency": { 2024-04-25T15:12:34.357 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.358 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.358 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.358 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.358 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmsnap_latency": { 2024-04-25T15:12:34.358 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 4, 2024-04-25T15:12:34.358 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.015568157, 2024-04-25T15:12:34.358 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.003892039 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: "req_rmxattr_latency": { 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setattr_latency": { 2024-04-25T15:12:34.359 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 1, 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.001058698, 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.001058698 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setdirlayout_latency": { 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.360 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setfilelock_latency": { 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setlayout_latency": { 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.361 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.362 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.362 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.362 INFO:teuthology.orchestra.run.smithi137.stdout: "req_setxattr_latency": { 2024-04-25T15:12:34.362 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 7, 2024-04-25T15:12:34.362 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.020705412, 2024-04-25T15:12:34.362 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.002957916 2024-04-25T15:12:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: "req_symlink_latency": { 2024-04-25T15:12:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.363 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: "req_unlink_latency": { 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_revoke_eviction": 0, 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: "cap_acquisition_throttle": 0, 2024-04-25T15:12:34.364 INFO:teuthology.orchestra.run.smithi137.stdout: "req_getvxattr_latency": { 2024-04-25T15:12:34.365 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.365 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.365 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.365 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.365 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.365 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_sessions": { 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "session_count": 2, 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "session_add": 5, 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "session_remove": 3, 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_open": 2, 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "sessions_stale": 0, 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "total_load": 27, 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "average_load": 13, 2024-04-25T15:12:34.366 INFO:teuthology.orchestra.run.smithi137.stdout: "avg_session_uptime": 596, 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: "mdthresh_evicted": 0 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:34.367 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:34.368 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:34.369 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:34.370 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:34.371 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 346995, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 139, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 792, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 9, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:34.372 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 8128, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 169, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T15:12:34.373 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 93946, 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 545, 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "objecter": { 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "op_active": 0, 2024-04-25T15:12:34.374 INFO:teuthology.orchestra.run.smithi137.stdout: "op_laggy": 0, 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send": 668, 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "op_send_bytes": 115959, 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "op_resend": 37, 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "op_reply": 668, 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "op_latency": { 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 668, 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 21.776627348, 2024-04-25T15:12:34.375 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.032599741 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: "op_inflight": 0, 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: "oplen_avg": { 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 668, 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 733 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: "op": 668, 2024-04-25T15:12:34.376 INFO:teuthology.orchestra.run.smithi137.stdout: "op_r": 0, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "op_w": 668, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "op_rmw": 0, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "op_pg": 0, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_stat": 1, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_create": 22, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_read": 0, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_write": 81, 2024-04-25T15:12:34.377 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writefull": 27, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_writesame": 0, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_append": 0, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_zero": 0, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_truncate": 0, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_delete": 506, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_mapext": 0, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_sparse_read": 0, 2024-04-25T15:12:34.378 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_clonerange": 0, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_getxattr": 0, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_setxattr": 44, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_cmpxattr": 0, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_rmxattr": 0, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_resetxattrs": 0, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_call": 0, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_watch": 0, 2024-04-25T15:12:34.379 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_notify": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_pgls_filter": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "osdop_other": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_active": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_send": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_resend": 0, 2024-04-25T15:12:34.380 INFO:teuthology.orchestra.run.smithi137.stdout: "linger_ping": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_active": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_send": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "poolop_resend": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_active": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_send": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "poolstat_resend": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_active": 0, 2024-04-25T15:12:34.381 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_send": 0, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "statfs_resend": 0, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "command_active": 0, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "command_send": 0, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "command_resend": 0, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "map_epoch": 97, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "map_full": 0, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "map_inc": 12, 2024-04-25T15:12:34.382 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_sessions": 12, 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_open": 12, 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_session_close": 0, 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_laggy": 0, 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_wr": 45, 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_rd": 0, 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_del": 7 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "oft": { 2024-04-25T15:12:34.383 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_objs": 1, 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_kv_pairs": 0, 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_updates": 0, 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: "omap_total_removes": 0 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: "osd-slow-ops": { 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: "slow_ops_count": 0 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.384 INFO:teuthology.orchestra.run.smithi137.stdout: "purge_queue": { 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops": 0, 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing": 0, 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executing_high_water": 0, 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed_ops": 0, 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_executed": 0, 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: "pq_item_in_journal": 0 2024-04-25T15:12:34.385 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 2600, 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 2204348, 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:34.386 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 2600, 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 2600, 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 2204348, 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.387 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_bytes": { 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T15:12:34.388 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 668, 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 115959, 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 668, 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 115959, 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.389 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.390 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.390 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.390 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.390 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-objecter_ops": { 2024-04-25T15:12:34.390 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:34.390 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 1024, 2024-04-25T15:12:34.390 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 668, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 668, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 668, 2024-04-25T15:12:34.391 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 668, 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle": { 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:34.392 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 0, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 0, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 0, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T15:12:34.393 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 0, 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 0, 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:34.394 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-write_buf_throttle-0x55ef03ec2500": { 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 3758096384, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 101, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 90524, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 101, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T15:12:34.395 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 81, 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 90524, 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.396 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:34.397 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T15:12:34.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:34 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:12:34.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:34 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:12:35.124 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T15:12:35.124 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T15:12:35.125 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T15:12:35.125 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T15:12:35.125 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T15:12:35.482 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:35 smithi137 ceph-mon[32439]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:35.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:35 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:35.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:35 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:35.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:35 smithi138 ceph-mon[32875]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:35.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:35 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:35.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:35 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:35 smithi094 ceph-mon[27862]: pgmap v323: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:35 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:35.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:35 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:36.046 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:12:36.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:36.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:36.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:36.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T15:12:36.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:36 smithi094 ceph-mon[27862]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:12:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:36.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:36.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T15:12:36.984 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:36 smithi137 ceph-mon[32439]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:12:37.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:37.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: pgmap v324: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:37.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:37.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.208 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.209 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.209 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.209 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:37.209 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi138 to 176.2M 2024-04-25T15:12:37.209 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:36 smithi138 ceph-mon[32875]: Unable to set osd_memory_target on smithi138 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:12:37.645 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T15:12:37.645 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.001024289, 2024-04-25T15:12:37.646 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000200613, 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:37.647 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 225, 2024-04-25T15:12:37.648 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 214, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1590464, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 131744, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.046921251, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.015097059, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.034529543, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.005713406, 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:37.649 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 213, 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.012761507, 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000059913 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:37.650 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1590464, 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 131744 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 6, 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 166, 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 3264, 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 185296, 2024-04-25T15:12:37.651 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012253255, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008095565, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001717027, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 164, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.028655050, 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000174725 2024-04-25T15:12:37.652 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 185296 2024-04-25T15:12:37.653 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:37.654 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:37.655 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:37.656 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:37.657 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148933, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 30, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:37.658 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:37.659 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T15:12:37.659 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T15:12:37.659 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:37.659 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:37.659 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:37.659 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:37.660 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:37.661 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 227, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1558750, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 227, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T15:12:37.662 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 227, 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1558750, 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:37.663 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T15:12:38.409 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T15:12:38.409 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T15:12:38.409 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T15:12:38.409 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T15:12:38.410 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T15:12:38.660 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:12:38.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:38 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:38 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:38 smithi137 ceph-mon[32439]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:38.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:38 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:38 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:38 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:38 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:38 smithi138 ceph-mon[32875]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:38.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:38 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:38 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:38 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:38 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:38 smithi094 ceph-mon[27862]: pgmap v325: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:38.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:38 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:38.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:38 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:39.722 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.723 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.724 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:12:39.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:12:39.725 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:39 smithi137 ceph-mon[32439]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:39.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.958 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.959 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.960 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.960 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.960 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:12:39.960 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:12:39.960 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:39 smithi138 ceph-mon[32875]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:39.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi094 to 176.4M 2024-04-25T15:12:39.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: Unable to set osd_memory_target on smithi094 to 185047654: error parsing value: Value '185047654' is below minimum 939524096 2024-04-25T15:12:39.964 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:12:39.965 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:12:39.966 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:12:39.966 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:39 smithi094 ceph-mon[27862]: from='mgr.15027 ' entity='mgr.x' 2024-04-25T15:12:40.279 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T15:12:40.279 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:40.279 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 4, 2024-04-25T15:12:40.279 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 4, 2024-04-25T15:12:40.279 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 2944, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 448, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.001355876, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000031771, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000525906, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:40.280 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 3, 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000629776, 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000209925 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:40.281 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 224, 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 213, 2024-04-25T15:12:40.282 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1590192, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 131104, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.045012052, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.013705996, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.033381844, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.005699880, 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:40.283 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 212, 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.012836529, 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000060549 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:40.284 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1590192, 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 131104 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 3, 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 164, 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 480, 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 262928, 2024-04-25T15:12:40.285 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 6, 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012135241, 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008200290, 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001688525, 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 160, 2024-04-25T15:12:40.286 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.019332885, 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000120830 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 262928 2024-04-25T15:12:40.287 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:40.288 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:40.289 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:40.290 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:40.291 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148995, 2024-04-25T15:12:40.292 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 31, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:40.293 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:40.294 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 227, 2024-04-25T15:12:40.295 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1558647, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 227, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 227, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1558647, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:40.296 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:40.297 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:40.297 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:40.297 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:40.297 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T15:12:40.949 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:40 smithi137 ceph-mon[32439]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T15:12:40.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:40 smithi137 ceph-mon[32439]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:12:40.950 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:40 smithi137 ceph-mon[32439]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:40 smithi138 ceph-mon[32875]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T15:12:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:40 smithi138 ceph-mon[32875]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:12:40.957 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:40 smithi138 ceph-mon[32875]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:40.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:40 smithi094 ceph-mon[27862]: Adjusting osd_memory_target on smithi137 to 176.2M 2024-04-25T15:12:40.962 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:40 smithi094 ceph-mon[27862]: Unable to set osd_memory_target on smithi137 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T15:12:40.963 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:40 smithi094 ceph-mon[27862]: pgmap v326: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:40.976 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-25T15:12:40.976 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-25T15:12:40.977 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-25T15:12:40.977 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T15:12:40.977 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T15:12:41.216 INFO:teuthology.orchestra.run.smithi137.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T15:12:42.853 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:42 smithi137 ceph-mon[32439]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:42.853 INFO:teuthology.orchestra.run.smithi137.stdout:{ 2024-04-25T15:12:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:42.854 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.000942833, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.000188595, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:42.855 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:42.856 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 233, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 220, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 1594496, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 134752, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.047793131, 2024-04-25T15:12:42.857 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.015184321, 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.034971678, 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.005784969, 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 220, 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.013675838, 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000062162 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:42.858 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 1594496, 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 134752 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:42.859 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_messages": 2, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages": 167, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_bytes": 185376, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_total_time": 0.012285587, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_send_time": 0.008587376, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_recv_time": 0.001333594, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:42.860 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 164, 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.022277639, 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000135839 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:42.861 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: "msgr_send_encrypted_bytes": 185376 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: "cct": { 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: "total_workers": 0, 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: "unhealthy_workers": 0 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.862 INFO:teuthology.orchestra.run.smithi137.stdout: "mempool": { 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:42.863 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:42.864 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:42.865 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_items": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:42.866 INFO:teuthology.orchestra.run.smithi137.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_bytes": 148732, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_anon_items": 31, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_bytes": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_items": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:42.867 INFO:teuthology.orchestra.run.smithi137.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_items": 0, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "pgmap_items": 0, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:42.868 INFO:teuthology.orchestra.run.smithi137.stdout: "mds_co_items": 2, 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_1_items": 0, 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "unittest_2_items": 0 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: }, 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "val": 0, 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "max": 104857600, 2024-04-25T15:12:42.869 INFO:teuthology.orchestra.run.smithi137.stdout: "get_started": 0, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "get": 231, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "get_sum": 1559270, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "get_or_fail_success": 231, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "take": 0, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "take_sum": 0, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "put": 231, 2024-04-25T15:12:42.870 INFO:teuthology.orchestra.run.smithi137.stdout: "put_sum": 1559270, 2024-04-25T15:12:42.871 INFO:teuthology.orchestra.run.smithi137.stdout: "wait": { 2024-04-25T15:12:42.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgcount": 0, 2024-04-25T15:12:42.871 INFO:teuthology.orchestra.run.smithi137.stdout: "sum": 0.000000000, 2024-04-25T15:12:42.871 INFO:teuthology.orchestra.run.smithi137.stdout: "avgtime": 0.000000000 2024-04-25T15:12:42.871 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:42.871 INFO:teuthology.orchestra.run.smithi137.stdout: } 2024-04-25T15:12:42.871 INFO:teuthology.orchestra.run.smithi137.stdout:} 2024-04-25T15:12:43.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:42 smithi138 ceph-mon[32875]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:43.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:42 smithi094 ceph-mon[27862]: pgmap v327: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:43.700 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T15:12:43.700 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T15:12:43.700 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T15:12:43.700 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T15:12:43.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T15:12:43.960 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:12:44.911 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:44 smithi094 ceph-mon[27862]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:45.045 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:44 smithi137 ceph-mon[32439]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:45.116 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:44 smithi138 ceph-mon[32875]: pgmap v328: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:45.708 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T15:12:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.001095736, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.000263658, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 230, 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 217, 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1593680, 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 132832, 2024-04-25T15:12:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.046604657, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.014823755, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.034267626, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.005502739, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 217, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.013690289, 2024-04-25T15:12:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000063088 2024-04-25T15:12:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1593680, 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 132832 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2, 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 165, 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:12:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 185024, 2024-04-25T15:12:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.011475760, 2024-04-25T15:12:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.007755942, 2024-04-25T15:12:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.001398350, 2024-04-25T15:12:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 162, 2024-04-25T15:12:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.029063479, 2024-04-25T15:12:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000179404 2024-04-25T15:12:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 185024 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T15:12:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 0, 2024-04-25T15:12:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T15:12:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T15:12:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 149017, 2024-04-25T15:12:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 31, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 0, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2, 2024-04-25T15:12:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T15:12:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T15:12:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 228, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1558880, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 228, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T15:12:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 228, 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1558880, 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:45.728 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T15:12:46.336 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-25T15:12:46.337 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-25T15:12:46.337 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-25T15:12:46.337 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T15:12:46.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T15:12:46.579 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:12:46.876 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:46 smithi138 ceph-mon[32875]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:47.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:46 smithi094 ceph-mon[27862]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:47.232 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:46 smithi137 ceph-mon[32439]: pgmap v329: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:48.274 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T15:12:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.000955595, 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.000119843, 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 231, 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 218, 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1593952, 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 133472, 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.047995066, 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.015271443, 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.035590493, 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.005665985, 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 218, 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.011987758, 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000054989 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1593952, 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 133472 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 165, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 185024, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.011142564, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.007359442, 2024-04-25T15:12:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.001395126, 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 162, 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.029411712, 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000181553 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 185024 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 0, 2024-04-25T15:12:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 148955, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 30, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 0, 2024-04-25T15:12:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T15:12:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2, 2024-04-25T15:12:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T15:12:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 229, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1559010, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 229, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 229, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1559010, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:48.293 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T15:12:48.853 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-25T15:12:48.853 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-25T15:12:48.854 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-25T15:12:48.854 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T15:12:48.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T15:12:49.091 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:12:49.480 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:49 smithi138 ceph-mon[32875]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:49.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:49 smithi094 ceph-mon[27862]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:49.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:49 smithi137 ceph-mon[32439]: pgmap v330: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:50.763 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T15:12:50.764 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:50.764 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:50.764 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:50.764 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.001211040, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.000305054, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:50.765 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:50.766 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2, 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 165, 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 320, 2024-04-25T15:12:50.767 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 185024, 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 5, 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.011246574, 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.007479602, 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.001346009, 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:50.768 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 162, 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.023288192, 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000143754 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.769 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 185024 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 230, 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 217, 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1593680, 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 132832, 2024-04-25T15:12:50.770 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.043835320, 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.013777630, 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.031664682, 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.005411684, 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 217, 2024-04-25T15:12:50.771 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.013396531, 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000061735 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1593680, 2024-04-25T15:12:50.772 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 132832 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 0, 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:50.773 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:50.774 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:50.775 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:50.776 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:50.777 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:50.777 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:50.777 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:50.777 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:50.777 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T15:12:50.777 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:50.777 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:50.778 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:50.778 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:50.778 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 148972, 2024-04-25T15:12:50.778 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 31, 2024-04-25T15:12:50.778 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:50.778 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:50.778 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 0, 2024-04-25T15:12:50.779 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:50.780 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 228, 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1558880, 2024-04-25T15:12:50.781 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 228, 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 228, 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1558880, 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:50.782 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:50.783 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:50.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:50.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:50.783 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T15:12:51.340 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:51 smithi138 ceph-mon[32875]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:51.342 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2024-04-25T15:12:51.342 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2024-04-25T15:12:51.342 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2024-04-25T15:12:51.342 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T15:12:51.343 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid f04d929c-0312-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T15:12:51.580 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T15:12:51.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:51 smithi094 ceph-mon[27862]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:51.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:51 smithi137 ceph-mon[32439]: pgmap v331: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:53.150 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T15:12:53.150 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T15:12:53.150 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 236, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 223, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1595312, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 136672, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 0, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.048329456, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.015043357, 2024-04-25T15:12:53.151 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.036304521, 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.005532078, 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 223, 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.013807301, 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000061916 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:53.152 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1595312, 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 136672 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T15:12:53.153 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 3, 2024-04-25T15:12:53.154 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 171, 2024-04-25T15:12:53.154 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 480, 2024-04-25T15:12:53.154 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 264336, 2024-04-25T15:12:53.154 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 6, 2024-04-25T15:12:53.154 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 3, 2024-04-25T15:12:53.154 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.011438612, 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.007761704, 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.001364363, 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 168, 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.022637948, 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000134749 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.155 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 264336 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T15:12:53.156 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 0, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 0, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 0, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 2, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.000214316, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T15:12:53.157 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.000161158, 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-04-25T15:12:53.158 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 0, 2024-04-25T15:12:53.159 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T15:12:53.160 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T15:12:53.161 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T15:12:53.162 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T15:12:53.163 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-04-25T15:12:53.163 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-04-25T15:12:53.163 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-04-25T15:12:53.163 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T15:12:53.163 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T15:12:53.163 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-04-25T15:12:53.163 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-04-25T15:12:53.164 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-04-25T15:12:53.164 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-04-25T15:12:53.164 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T15:12:53.164 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-04-25T15:12:53.164 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T15:12:53.164 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-04-25T15:12:53.164 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 148692, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 30, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-04-25T15:12:53.165 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-04-25T15:12:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-04-25T15:12:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 0, 2024-04-25T15:12:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 0, 2024-04-25T15:12:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T15:12:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-04-25T15:12:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-04-25T15:12:53.166 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-04-25T15:12:53.167 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 928, 2024-04-25T15:12:53.167 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2, 2024-04-25T15:12:53.167 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-04-25T15:12:53.167 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-04-25T15:12:53.167 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-04-25T15:12:53.167 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 235, 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1559687, 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-04-25T15:12:53.168 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 235, 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 235, 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1559687, 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-04-25T15:12:53.169 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-04-25T15:12:53.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:53.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T15:12:53.170 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T15:12:53.427 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:53 smithi138 ceph-mon[32875]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:53.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:53 smithi094 ceph-mon[27862]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:53.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:53 smithi137 ceph-mon[32439]: pgmap v332: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:53.774 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T15:12:53.774 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T15:12:53.774 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T15:12:53.774 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T15:12:53.785 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T15:12:53.785 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T15:12:53.786 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T15:12:53.847 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-04-25T15:12:53.847 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T15:12:53.900 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:12:54.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 -- 172.21.15.94:0/636228926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c1083f0 msgr2=0x7f229c108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 --2- 172.21.15.94:0/636228926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c1083f0 0x7f229c108850 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2288009860 tx=0x7f228802a3d0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 -- 172.21.15.94:0/636228926 shutdown_connections 2024-04-25T15:12:54.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 --2- 172.21.15.94:0/636228926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c108d90 0x7f229c115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 --2- 172.21.15.94:0/636228926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c1083f0 0x7f229c108850 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 --2- 172.21.15.94:0/636228926 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f229c10e460 0x7f229c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 -- 172.21.15.94:0/636228926 >> 172.21.15.94:0/636228926 conn(0x7f229c077ad0 msgr2=0x7f229c079ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:54.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 -- 172.21.15.94:0/636228926 shutdown_connections 2024-04-25T15:12:54.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.106+0000 7f22a3200640 1 -- 172.21.15.94:0/636228926 wait complete. 2024-04-25T15:12:54.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.107+0000 7f22a3200640 1 Processor -- start 2024-04-25T15:12:54.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a3200640 1 -- start start 2024-04-25T15:12:54.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a3200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c1083f0 0x7f229c1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c108d90 0x7f229c1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a3200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f229c10e460 0x7f229c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a3200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f229c118620 con 0x7f229c1083f0 2024-04-25T15:12:54.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a3200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1184a0 con 0x7f229c10e460 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f229c1187a0 con 0x7f229c108d90 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c1083f0 0x7f229c1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c1083f0 0x7f229c1a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34650/0 (socket says 172.21.15.94:34650) 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a1e00640 1 -- 172.21.15.94:0/1711889719 learned_addr learned my addr 172.21.15.94:0/1711889719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.108+0000 7f22a2800640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f229c10e460 0x7f229c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a1400640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c108d90 0x7f229c1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a1e00640 1 -- 172.21.15.94:0/1711889719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c108d90 msgr2=0x7f229c1a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a1e00640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c108d90 0x7f229c1a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a1e00640 1 -- 172.21.15.94:0/1711889719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f229c10e460 msgr2=0x7f229c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a1e00640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f229c10e460 0x7f229c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a1e00640 1 -- 172.21.15.94:0/1711889719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c19f040 con 0x7f229c1083f0 2024-04-25T15:12:54.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a2800640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f229c10e460 0x7f229c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:12:54.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f22a1e00640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c1083f0 0x7f229c1a4710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f229000e9f0 tx=0x7f229000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:54.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.109+0000 7f2296a00640 1 -- 172.21.15.94:0/1711889719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229000cd80 con 0x7f229c1083f0 2024-04-25T15:12:54.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.110+0000 7f2296a00640 1 -- 172.21.15.94:0/1711889719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22900108e0 con 0x7f229c1083f0 2024-04-25T15:12:54.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.110+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c19f330 con 0x7f229c1083f0 2024-04-25T15:12:54.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.110+0000 7f2296a00640 1 -- 172.21.15.94:0/1711889719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290018980 con 0x7f229c1083f0 2024-04-25T15:12:54.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.110+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c1ab5d0 con 0x7f229c1083f0 2024-04-25T15:12:54.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.111+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c0690e0 con 0x7f229c1083f0 2024-04-25T15:12:54.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.113+0000 7f2296a00640 1 -- 172.21.15.94:0/1711889719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2290010430 con 0x7f229c1083f0 2024-04-25T15:12:54.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.114+0000 7f2296a00640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f22840a3d20 0x7f22840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.114+0000 7f2296a00640 1 -- 172.21.15.94:0/1711889719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f22900d0b30 con 0x7f229c1083f0 2024-04-25T15:12:54.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.115+0000 7f22a1400640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f22840a3d20 0x7f22840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.115+0000 7f22a1400640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f22840a3d20 0x7f22840a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f229c1a00b0 tx=0x7f2288005c90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:54.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.119+0000 7f2296a00640 1 -- 172.21.15.94:0/1711889719 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f229009cb10 con 0x7f229c1083f0 2024-04-25T15:12:54.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.324+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f229c1102c0 con 0x7f229c1083f0 2024-04-25T15:12:54.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.328+0000 7f2296a00640 1 -- 172.21.15.94:0/1711889719 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23151 (secure 0 0 0) 0x7f22900a0e10 con 0x7f229c1083f0 2024-04-25T15:12:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:12:54.331 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":97,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:12:17.235043+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/702744035":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3354212797":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6801/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1994260887":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/2523030980":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6800/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:12:54.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f22840a3d20 msgr2=0x7f22840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f22840a3d20 0x7f22840a61e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f229c1a00b0 tx=0x7f2288005c90 comp rx=0 tx=0).stop 2024-04-25T15:12:54.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c1083f0 msgr2=0x7f229c1a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c1083f0 0x7f229c1a4710 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f229000e9f0 tx=0x7f229000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 shutdown_connections 2024-04-25T15:12:54.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f22840a3d20 0x7f22840a61e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f229c10e460 0x7f229c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229c108d90 0x7f229c1a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 --2- 172.21.15.94:0/1711889719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f229c1083f0 0x7f229c1a4710 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 >> 172.21.15.94:0/1711889719 conn(0x7f229c077ad0 msgr2=0x7f229c0791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:54.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 shutdown_connections 2024-04-25T15:12:54.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.331+0000 7f22a3200640 1 -- 172.21.15.94:0/1711889719 wait complete. 2024-04-25T15:12:54.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T14:54:45.505952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T14:59:43.356930+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T14:59:44.767550+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 96, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T15:12:54.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:12:54.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 -- 172.21.15.94:0/3697325779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 msgr2=0x7f4144105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 --2- 172.21.15.94:0/3697325779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 0x7f4144105200 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f412c009900 tx=0x7f412c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 -- 172.21.15.94:0/3697325779 shutdown_connections 2024-04-25T15:12:54.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 --2- 172.21.15.94:0/3697325779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f414410af60 0x7f414410d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 --2- 172.21.15.94:0/3697325779 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 0x7f4144105200 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 --2- 172.21.15.94:0/3697325779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4144068400 0x7f4144104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 -- 172.21.15.94:0/3697325779 >> 172.21.15.94:0/3697325779 conn(0x7f4144075c50 msgr2=0x7f4144076060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 -- 172.21.15.94:0/3697325779 shutdown_connections 2024-04-25T15:12:54.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.555+0000 7f4143e00640 1 -- 172.21.15.94:0/3697325779 wait complete. 2024-04-25T15:12:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.556+0000 7f4143e00640 1 Processor -- start 2024-04-25T15:12:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.556+0000 7f4143e00640 1 -- start start 2024-04-25T15:12:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4144068400 0x7f41441986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 0x7f4144198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f414410af60 0x7f414419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f4144111020 con 0x7f414410af60 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f4144110ea0 con 0x7f4144100e90 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41441111a0 con 0x7f4144068400 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f414410af60 0x7f414419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4142000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 0x7f4144198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4142a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4144068400 0x7f41441986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f414410af60 0x7f414419cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34670/0 (socket says 172.21.15.94:34670) 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4142000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 0x7f4144198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58122/0 (socket says 172.21.15.94:58122) 2024-04-25T15:12:54.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.557+0000 7f4143400640 1 -- 172.21.15.94:0/4129230325 learned_addr learned my addr 172.21.15.94:0/4129230325 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4142a00640 1 -- 172.21.15.94:0/4129230325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 msgr2=0x7f4144198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4142a00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 0x7f4144198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4142a00640 1 -- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f414410af60 msgr2=0x7f414419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4142a00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f414410af60 0x7f414419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4142a00640 1 -- 172.21.15.94:0/4129230325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414419d6e0 con 0x7f4144068400 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4142a00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4144068400 0x7f41441986f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f413400e9c0 tx=0x7f413400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f413b400640 1 -- 172.21.15.94:0/4129230325 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400ccc0 con 0x7f4144068400 2024-04-25T15:12:54.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f413b400640 1 -- 172.21.15.94:0/4129230325 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41340035f0 con 0x7f4144068400 2024-04-25T15:12:54.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414419d9d0 con 0x7f4144068400 2024-04-25T15:12:54.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f413b400640 1 -- 172.21.15.94:0/4129230325 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413400ce60 con 0x7f4144068400 2024-04-25T15:12:54.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.558+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41441a5260 con 0x7f4144068400 2024-04-25T15:12:54.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.559+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41440690e0 con 0x7f4144068400 2024-04-25T15:12:54.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.562+0000 7f413b400640 1 -- 172.21.15.94:0/4129230325 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4134004000 con 0x7f4144068400 2024-04-25T15:12:54.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.563+0000 7f413b400640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f41280a3d20 0x7f41280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:54.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.563+0000 7f413b400640 1 -- 172.21.15.94:0/4129230325 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f41340d0b90 con 0x7f4144068400 2024-04-25T15:12:54.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.563+0000 7f4142000640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f41280a3d20 0x7f41280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:54.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.563+0000 7f4142000640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f41280a3d20 0x7f41280a61e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f412c02a9e0 tx=0x7f412c005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:54.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.565+0000 7f413b400640 1 -- 172.21.15.94:0/4129230325 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f413409d1c0 con 0x7f4144068400 2024-04-25T15:12:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.767+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41441a5430 con 0x7f4144068400 2024-04-25T15:12:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.767+0000 7f413b400640 1 -- 172.21.15.94:0/4129230325 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f413409cac0 con 0x7f4144068400 2024-04-25T15:12:54.768 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f41280a3d20 msgr2=0x7f41280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f41280a3d20 0x7f41280a61e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f412c02a9e0 tx=0x7f412c005a20 comp rx=0 tx=0).stop 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4144068400 msgr2=0x7f41441986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4144068400 0x7f41441986f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f413400e9c0 tx=0x7f413400ee80 comp rx=0 tx=0).stop 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 shutdown_connections 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f41280a3d20 0x7f41280a61e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f414410af60 0x7f414419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f4144100e90 0x7f4144198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 --2- 172.21.15.94:0/4129230325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4144068400 0x7f41441986f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 >> 172.21.15.94:0/4129230325 conn(0x7f4144075c50 msgr2=0x7f4144103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 shutdown_connections 2024-04-25T15:12:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.770+0000 7f4143e00640 1 -- 172.21.15.94:0/4129230325 wait complete. 2024-04-25T15:12:54.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T15:12:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 -- 172.21.15.94:0/938842709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 msgr2=0x7f146c1152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 --2- 172.21.15.94:0/938842709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1152b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1464009ca0 tx=0x7f146402a950 comp rx=0 tx=0).stop 2024-04-25T15:12:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 -- 172.21.15.94:0/938842709 shutdown_connections 2024-04-25T15:12:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 --2- 172.21.15.94:0/938842709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1152b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 --2- 172.21.15.94:0/938842709 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c1084f0 0x7f146c108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 --2- 172.21.15.94:0/938842709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f146c10d3c0 0x7f146c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 -- 172.21.15.94:0/938842709 >> 172.21.15.94:0/938842709 conn(0x7f146c077d70 msgr2=0x7f146c106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 -- 172.21.15.94:0/938842709 shutdown_connections 2024-04-25T15:12:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.996+0000 7f1472800640 1 -- 172.21.15.94:0/938842709 wait complete. 2024-04-25T15:12:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 Processor -- start 2024-04-25T15:12:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 -- start start 2024-04-25T15:12:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c1084f0 0x7f146c1a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f146c10d3c0 0x7f146c19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f146c1186b0 con 0x7f146c108e90 2024-04-25T15:12:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f146c118530 con 0x7f146c1084f0 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.997+0000 7f1472800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f146c118830 con 0x7f146c10d3c0 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34692/0 (socket says 172.21.15.94:34692) 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 -- 172.21.15.94:0/2294263301 learned_addr learned my addr 172.21.15.94:0/2294263301 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1471e00640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f146c10d3c0 0x7f146c19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1471400640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c1084f0 0x7f146c1a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 -- 172.21.15.94:0/2294263301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f146c10d3c0 msgr2=0x7f146c19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f146c10d3c0 0x7f146c19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 -- 172.21.15.94:0/2294263301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c1084f0 msgr2=0x7f146c1a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c1084f0 0x7f146c1a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.998+0000 7f1470a00640 1 -- 172.21.15.94:0/2294263301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c19f170 con 0x7f146c108e90 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.999+0000 7f1470a00640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1a4d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f145c00ea60 tx=0x7f145c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.999+0000 7f146a000640 1 -- 172.21.15.94:0/2294263301 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c00cde0 con 0x7f146c108e90 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.999+0000 7f146a000640 1 -- 172.21.15.94:0/2294263301 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f145c0035f0 con 0x7f146c108e90 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.999+0000 7f146a000640 1 -- 172.21.15.94:0/2294263301 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145c010690 con 0x7f146c108e90 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.999+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c19f460 con 0x7f146c108e90 2024-04-25T15:12:55.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:54.999+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1ab700 con 0x7f146c108e90 2024-04-25T15:12:55.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.000+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f146c0690e0 con 0x7f146c108e90 2024-04-25T15:12:55.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.002+0000 7f146a000640 1 -- 172.21.15.94:0/2294263301 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f145c0037a0 con 0x7f146c108e90 2024-04-25T15:12:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.003+0000 7f146a000640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f14540a3d20 0x7f14540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.004+0000 7f146a000640 1 -- 172.21.15.94:0/2294263301 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f145c0d0b10 con 0x7f146c108e90 2024-04-25T15:12:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.004+0000 7f1471400640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f14540a3d20 0x7f14540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.004+0000 7f1471400640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f14540a3d20 0x7f14540a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f146c19fb30 tx=0x7f1460005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.008+0000 7f146a000640 1 -- 172.21.15.94:0/2294263301 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f145c09d140 con 0x7f146c108e90 2024-04-25T15:12:55.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.215+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f146c110960 con 0x7f146c108e90 2024-04-25T15:12:55.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.216+0000 7f146a000640 1 -- 172.21.15.94:0/2294263301 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f145c09ca40 con 0x7f146c108e90 2024-04-25T15:12:55.217 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-04-25T15:12:55.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f14540a3d20 msgr2=0x7f14540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f14540a3d20 0x7f14540a61e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f146c19fb30 tx=0x7f1460005f90 comp rx=0 tx=0).stop 2024-04-25T15:12:55.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 msgr2=0x7f146c1a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1a4d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f145c00ea60 tx=0x7f145c00ef20 comp rx=0 tx=0).stop 2024-04-25T15:12:55.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 shutdown_connections 2024-04-25T15:12:55.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f14540a3d20 0x7f14540a61e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f146c10d3c0 0x7f146c19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f146c108e90 0x7f146c1a4d80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 --2- 172.21.15.94:0/2294263301 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f146c1084f0 0x7f146c1a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 >> 172.21.15.94:0/2294263301 conn(0x7f146c077d70 msgr2=0x7f146c078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:55.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 shutdown_connections 2024-04-25T15:12:55.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.219+0000 7f1472800640 1 -- 172.21.15.94:0/2294263301 wait complete. 2024-04-25T15:12:55.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T15:12:55.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 -- 172.21.15.94:0/576502059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec112c90 msgr2=0x7f7dec115080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 --2- 172.21.15.94:0/576502059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec112c90 0x7f7dec115080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009ca0 tx=0x7f7de402a8c0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 -- 172.21.15.94:0/576502059 shutdown_connections 2024-04-25T15:12:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 --2- 172.21.15.94:0/576502059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec112c90 0x7f7dec115080 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 --2- 172.21.15.94:0/576502059 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dec076da0 0x7f7dec112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 --2- 172.21.15.94:0/576502059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dec076480 0x7f7dec076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 -- 172.21.15.94:0/576502059 >> 172.21.15.94:0/576502059 conn(0x7f7dec0ffac0 msgr2=0x7f7dec101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 -- 172.21.15.94:0/576502059 shutdown_connections 2024-04-25T15:12:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.443+0000 7f7df0e00640 1 -- 172.21.15.94:0/576502059 wait complete. 2024-04-25T15:12:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 Processor -- start 2024-04-25T15:12:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 -- start start 2024-04-25T15:12:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dec076480 0x7f7dec1a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec076da0 0x7f7dec1a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dec112c90 0x7f7dec19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec1186d0 con 0x7f7dec076da0 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec118550 con 0x7f7dec112c90 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7df0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec118850 con 0x7f7dec076480 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec076da0 0x7f7dec1a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7deaa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec076da0 0x7f7dec1a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34712/0 (socket says 172.21.15.94:34712) 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.444+0000 7f7deb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dec076480 0x7f7dec1a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deaa00640 1 -- 172.21.15.94:0/2655035313 learned_addr learned my addr 172.21.15.94:0/2655035313 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7debe00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dec112c90 0x7f7dec19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deaa00640 1 -- 172.21.15.94:0/2655035313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dec076480 msgr2=0x7f7dec1a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deaa00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dec076480 0x7f7dec1a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deaa00640 1 -- 172.21.15.94:0/2655035313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dec112c90 msgr2=0x7f7dec19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deaa00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dec112c90 0x7f7dec19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deaa00640 1 -- 172.21.15.94:0/2655035313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec19f170 con 0x7f7dec076da0 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deb400640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dec076480 0x7f7dec1a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7deaa00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec076da0 0x7f7dec1a4d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800e9f0 tx=0x7f7dd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.445+0000 7f7de3e00640 1 -- 172.21.15.94:0/2655035313 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd800cd80 con 0x7f7dec076da0 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.446+0000 7f7de3e00640 1 -- 172.21.15.94:0/2655035313 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd80108e0 con 0x7f7dec076da0 2024-04-25T15:12:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.446+0000 7f7de3e00640 1 -- 172.21.15.94:0/2655035313 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8018980 con 0x7f7dec076da0 2024-04-25T15:12:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.446+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec19f460 con 0x7f7dec076da0 2024-04-25T15:12:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.446+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1ab700 con 0x7f7dec076da0 2024-04-25T15:12:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.447+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dec0690e0 con 0x7f7dec076da0 2024-04-25T15:12:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.449+0000 7f7de3e00640 1 -- 172.21.15.94:0/2655035313 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7dd8010430 con 0x7f7dec076da0 2024-04-25T15:12:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.451+0000 7f7de3e00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7dd00a3da0 0x7f7dd00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.451+0000 7f7de3e00640 1 -- 172.21.15.94:0/2655035313 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f7dd80d0a40 con 0x7f7dec076da0 2024-04-25T15:12:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.451+0000 7f7deb400640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7dd00a3da0 0x7f7dd00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.452+0000 7f7deb400640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7dd00a3da0 0x7f7dd00a6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7dec19fb30 tx=0x7f7ddc005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:55.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.455+0000 7f7de3e00640 1 -- 172.21.15.94:0/2655035313 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7dd809ca20 con 0x7f7dec076da0 2024-04-25T15:12:55.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.659+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7dec10d890 con 0x7f7dec076da0 2024-04-25T15:12:55.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.660+0000 7f7de3e00640 1 -- 172.21.15.94:0/2655035313 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f7dd80a0d20 con 0x7f7dec076da0 2024-04-25T15:12:55.661 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-04-25T15:12:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7dd00a3da0 msgr2=0x7f7dd00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7dd00a3da0 0x7f7dd00a6260 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7dec19fb30 tx=0x7f7ddc005f90 comp rx=0 tx=0).stop 2024-04-25T15:12:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec076da0 msgr2=0x7f7dec1a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec076da0 0x7f7dec1a4d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7dd800e9f0 tx=0x7f7dd800eeb0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 shutdown_connections 2024-04-25T15:12:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f7dd00a3da0 0x7f7dd00a6260 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f7dec112c90 0x7f7dec19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7dec076da0 0x7f7dec1a4d80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 --2- 172.21.15.94:0/2655035313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dec076480 0x7f7dec1a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 >> 172.21.15.94:0/2655035313 conn(0x7f7dec0ffac0 msgr2=0x7f7dec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 shutdown_connections 2024-04-25T15:12:55.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.663+0000 7f7df0e00640 1 -- 172.21.15.94:0/2655035313 wait complete. 2024-04-25T15:12:55.680 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:55 smithi094 ceph-mon[27862]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:55.680 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:55 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1711889719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:55.680 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:55 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4129230325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:12:55.680 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:55 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2294263301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:12:55.683 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T15:12:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:55 smithi138 ceph-mon[32875]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1711889719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4129230325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:12:55.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:55 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2294263301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:12:55.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:55 smithi137 ceph-mon[32439]: pgmap v333: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:55.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:55 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1711889719' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:55.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:55 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4129230325' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:12:55.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:55 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2294263301' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T15:12:55.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 -- 172.21.15.94:0/719374532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 msgr2=0x7f84f41047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 --2- 172.21.15.94:0/719374532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41047e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009900 tx=0x7f84e402a3f0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 -- 172.21.15.94:0/719374532 shutdown_connections 2024-04-25T15:12:55.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 --2- 172.21.15.94:0/719374532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f41094d0 0x7f84f41110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 --2- 172.21.15.94:0/719374532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84f4104db0 0x7f84f4108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 --2- 172.21.15.94:0/719374532 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41047e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 -- 172.21.15.94:0/719374532 >> 172.21.15.94:0/719374532 conn(0x7f84f40ffce0 msgr2=0x7f84f4102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:55.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 -- 172.21.15.94:0/719374532 shutdown_connections 2024-04-25T15:12:55.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.890+0000 7f84f9000640 1 -- 172.21.15.94:0/719374532 wait complete. 2024-04-25T15:12:55.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 Processor -- start 2024-04-25T15:12:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 -- start start 2024-04-25T15:12:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f41094d0 0x7f84f41a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84f419a860 0x7f84f419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4114480 con 0x7f84f41094d0 2024-04-25T15:12:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4114300 con 0x7f84f4104400 2024-04-25T15:12:55.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.891+0000 7f84f9000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4114600 con 0x7f84f419a860 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f41094d0 0x7f84f41a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:58180/0 (socket says 172.21.15.94:58180) 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f41094d0 0x7f84f41a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34724/0 (socket says 172.21.15.94:34724) 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 -- 172.21.15.94:0/890080307 learned_addr learned my addr 172.21.15.94:0/890080307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f2800640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84f419a860 0x7f84f419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 -- 172.21.15.94:0/890080307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84f419a860 msgr2=0x7f84f419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84f419a860 0x7f84f419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 -- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f41094d0 msgr2=0x7f84f41a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f41094d0 0x7f84f41a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 -- 172.21.15.94:0/890080307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f419b380 con 0x7f84f4104400 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.892+0000 7f84f1e00640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41a06b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005ab0 tx=0x7f84e4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.893+0000 7f84eaa00640 1 -- 172.21.15.94:0/890080307 <== mon.2 v2:172.21.15.137:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e403f070 con 0x7f84f4104400 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.893+0000 7f84eaa00640 1 -- 172.21.15.94:0/890080307 <== mon.2 v2:172.21.15.137:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84e4031b60 con 0x7f84f4104400 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.893+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f419b610 con 0x7f84f4104400 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.893+0000 7f84eaa00640 1 -- 172.21.15.94:0/890080307 <== mon.2 v2:172.21.15.137:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e4035030 con 0x7f84f4104400 2024-04-25T15:12:55.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.893+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f41a7860 con 0x7f84f4104400 2024-04-25T15:12:55.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.894+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f40691c0 con 0x7f84f4104400 2024-04-25T15:12:55.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.896+0000 7f84eaa00640 1 -- 172.21.15.94:0/890080307 <== mon.2 v2:172.21.15.137:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84e4042070 con 0x7f84f4104400 2024-04-25T15:12:55.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.897+0000 7f84eaa00640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f84d80a3da0 0x7f84d80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:55.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.897+0000 7f84eaa00640 1 -- 172.21.15.94:0/890080307 <== mon.2 v2:172.21.15.137:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f84e40f16d0 con 0x7f84f4104400 2024-04-25T15:12:55.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.898+0000 7f84f1400640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f84d80a3da0 0x7f84d80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:55.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.898+0000 7f84f1400640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f84d80a3da0 0x7f84d80a6260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84f410aa10 tx=0x7f84e0005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:55.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:55.900+0000 7f84eaa00640 1 -- 172.21.15.94:0/890080307 <== mon.2 v2:172.21.15.137:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84e40bdd00 con 0x7f84f4104400 2024-04-25T15:12:56.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.119+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f84f410c700 con 0x7f84f4104400 2024-04-25T15:12:56.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.120+0000 7f84eaa00640 1 -- 172.21.15.94:0/890080307 <== mon.2 v2:172.21.15.137:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v97) ==== 81+0+940 (secure 0 0 0) 0x7f84e40bd600 con 0x7f84f4104400 2024-04-25T15:12:56.120 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/702744035 2024-04-26T15:02:09.415076+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3354212797 2024-04-26T15:02:09.415076+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2485357518 2024-04-26T15:02:09.415076+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1994260887 2024-04-26T15:02:09.415076+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3440614830 2024-04-26T14:52:01.342497+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1490569056 2024-04-26T14:52:01.342497+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/1217751898 2024-04-26T14:52:01.342497+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/1217751898 2024-04-26T14:52:01.342497+0000 2024-04-25T15:12:56.121 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2523030980 2024-04-26T15:02:09.415076+0000 2024-04-25T15:12:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/17546499 2024-04-26T14:51:27.240380+0000 2024-04-25T15:12:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2485357518 2024-04-26T15:02:09.415076+0000 2024-04-25T15:12:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1137535670 2024-04-26T14:51:27.240380+0000 2024-04-25T15:12:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2519672579 2024-04-26T14:51:27.240380+0000 2024-04-25T15:12:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/312144263 2024-04-26T14:51:27.240380+0000 2024-04-25T15:12:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/17546499 2024-04-26T14:51:27.240380+0000 2024-04-25T15:12:56.122 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3116385060 2024-04-26T14:52:01.342497+0000 2024-04-25T15:12:56.123 INFO:teuthology.orchestra.run.smithi094.stderr:listed 16 entries 2024-04-25T15:12:56.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f84d80a3da0 msgr2=0x7f84d80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f84d80a3da0 0x7f84d80a6260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f84f410aa10 tx=0x7f84e0005f90 comp rx=0 tx=0).stop 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 msgr2=0x7f84f41a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41a06b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84e4005ab0 tx=0x7f84e4003c10 comp rx=0 tx=0).stop 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 shutdown_connections 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f84d80a3da0 0x7f84d80a6260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84f419a860 0x7f84f419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84f41094d0 0x7f84f41a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 --2- 172.21.15.94:0/890080307 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f84f4104400 0x7f84f41a06b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.122+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 >> 172.21.15.94:0/890080307 conn(0x7f84f40ffce0 msgr2=0x7f84f4101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:56.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.123+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 shutdown_connections 2024-04-25T15:12:56.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.123+0000 7f84f9000640 1 -- 172.21.15.94:0/890080307 wait complete. 2024-04-25T15:12:56.142 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T15:12:56.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:12:56.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T15:12:56.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:12:56.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:12:56.238 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T15:12:56.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:12:56.307 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-04-25T15:12:56.359 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T15:12:56.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:12:56.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:12:56.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:56 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2655035313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:12:56.430 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:56 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/890080307' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T15:12:56.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:12:56.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-04-25T15:12:56.572 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T15:12:56.583 INFO:tasks.cephadm:Teardown begin 2024-04-25T15:12:56.583 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T15:12:56.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:12:56.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:56 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2655035313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:12:56.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:56 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/890080307' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T15:12:56.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:56 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2655035313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T15:12:56.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:56 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/890080307' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T15:12:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 -- 172.21.15.94:0/2668622419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90076da0 msgr2=0x7f2d90112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 --2- 172.21.15.94:0/2668622419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90076da0 0x7f2d90112880 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009930 tx=0x7f2d7c02a420 comp rx=0 tx=0).stop 2024-04-25T15:12:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 -- 172.21.15.94:0/2668622419 shutdown_connections 2024-04-25T15:12:56.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 --2- 172.21.15.94:0/2668622419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90112dc0 0x7f2d901151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 --2- 172.21.15.94:0/2668622419 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90076da0 0x7f2d90112880 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 --2- 172.21.15.94:0/2668622419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d90076480 0x7f2d90076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 -- 172.21.15.94:0/2668622419 >> 172.21.15.94:0/2668622419 conn(0x7f2d900ffc50 msgr2=0x7f2d90102070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 -- 172.21.15.94:0/2668622419 shutdown_connections 2024-04-25T15:12:56.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.764+0000 7f2d96e00640 1 -- 172.21.15.94:0/2668622419 wait complete. 2024-04-25T15:12:56.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.765+0000 7f2d96e00640 1 Processor -- start 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.765+0000 7f2d96e00640 1 -- start start 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d96e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90076480 0x7f2d9019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d96e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d90076da0 0x7f2d9019f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d96e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90112dc0 0x7f2d901a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d96e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901187e0 con 0x7f2d90076480 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d96e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90118660 con 0x7f2d90112dc0 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d96e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90118960 con 0x7f2d90076da0 2024-04-25T15:12:56.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d8fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90076480 0x7f2d9019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d8fe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90076480 0x7f2d9019ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34738/0 (socket says 172.21.15.94:34738) 2024-04-25T15:12:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d8fe00640 1 -- 172.21.15.94:0/1461451493 learned_addr learned my addr 172.21.15.94:0/1461451493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d8f400640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d90076da0 0x7f2d9019f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.766+0000 7f2d94e00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90112dc0 0x7f2d901a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8fe00640 1 -- 172.21.15.94:0/1461451493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d90076da0 msgr2=0x7f2d9019f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8fe00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d90076da0 0x7f2d9019f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8fe00640 1 -- 172.21.15.94:0/1461451493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90112dc0 msgr2=0x7f2d901a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8fe00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90112dc0 0x7f2d901a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8fe00640 1 -- 172.21.15.94:0/1461451493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d901a3e10 con 0x7f2d90076480 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8f400640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d90076da0 0x7f2d9019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8fe00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90076480 0x7f2d9019ee70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000e9f0 tx=0x7f2d8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8cc00640 1 -- 172.21.15.94:0/1461451493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000cd80 con 0x7f2d90076480 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8cc00640 1 -- 172.21.15.94:0/1461451493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d800054d0 con 0x7f2d90076480 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d901a4100 con 0x7f2d90076480 2024-04-25T15:12:56.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d8cc00640 1 -- 172.21.15.94:0/1461451493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80010690 con 0x7f2d90076480 2024-04-25T15:12:56.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.767+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d901ab990 con 0x7f2d90076480 2024-04-25T15:12:56.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.768+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d900691c0 con 0x7f2d90076480 2024-04-25T15:12:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.770+0000 7f2d8cc00640 1 -- 172.21.15.94:0/1461451493 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d80004e90 con 0x7f2d90076480 2024-04-25T15:12:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.772+0000 7f2d8cc00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2d740a3d20 0x7f2d740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.772+0000 7f2d8cc00640 1 -- 172.21.15.94:0/1461451493 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f2d800d0c60 con 0x7f2d90076480 2024-04-25T15:12:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.772+0000 7f2d8f400640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2d740a3d20 0x7f2d740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:56.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.773+0000 7f2d8f400640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2d740a3d20 0x7f2d740a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009900 tx=0x7f2d7c000bc0 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:56.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:56.776+0000 7f2d8cc00640 1 -- 172.21.15.94:0/1461451493 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d8009d290 con 0x7f2d90076480 2024-04-25T15:12:57.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.019+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d9010dfb0 con 0x7f2d90076480 2024-04-25T15:12:57.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.021+0000 7f2d8cc00640 1 -- 172.21.15.94:0/1461451493 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f2d8009cb90 con 0x7f2d90076480 2024-04-25T15:12:57.022 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:12:57.023 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:12:57.023 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:12:57.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2d740a3d20 msgr2=0x7f2d740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2d740a3d20 0x7f2d740a61e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009900 tx=0x7f2d7c000bc0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90076480 msgr2=0x7f2d9019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90076480 0x7f2d9019ee70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2d8000e9f0 tx=0x7f2d8000edf0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 shutdown_connections 2024-04-25T15:12:57.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2d740a3d20 0x7f2d740a61e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2d90112dc0 0x7f2d901a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d90076da0 0x7f2d9019f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 --2- 172.21.15.94:0/1461451493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d90076480 0x7f2d9019ee70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 >> 172.21.15.94:0/1461451493 conn(0x7f2d900ffc50 msgr2=0x7f2d90113540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:57.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 shutdown_connections 2024-04-25T15:12:57.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.024+0000 7f2d96e00640 1 -- 172.21.15.94:0/1461451493 wait complete. 2024-04-25T15:12:57.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:12:57.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 -- 172.21.15.94:0/353027525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a181041e0 msgr2=0x7f3a181045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 --2- 172.21.15.94:0/353027525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a181041e0 0x7f3a181045c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009900 tx=0x7f3a0c02a3f0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 -- 172.21.15.94:0/353027525 shutdown_connections 2024-04-25T15:12:57.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 --2- 172.21.15.94:0/353027525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181092b0 0x7f3a18110e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 --2- 172.21.15.94:0/353027525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a18104b90 0x7f3a18108bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 --2- 172.21.15.94:0/353027525 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a181041e0 0x7f3a181045c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 -- 172.21.15.94:0/353027525 >> 172.21.15.94:0/353027525 conn(0x7f3a180ffa20 msgr2=0x7f3a18101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:57.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 -- 172.21.15.94:0/353027525 shutdown_connections 2024-04-25T15:12:57.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.241+0000 7f3a1f200640 1 -- 172.21.15.94:0/353027525 wait complete. 2024-04-25T15:12:57.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.242+0000 7f3a1f200640 1 Processor -- start 2024-04-25T15:12:57.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.242+0000 7f3a1f200640 1 -- start start 2024-04-25T15:12:57.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1f200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181041e0 0x7f3a181a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a181092b0 0x7f3a181a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1f200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a1819a600 0x7f3a1819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1f200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f3a18114240 con 0x7f3a181041e0 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1f200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181140c0 con 0x7f3a1819a600 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a181143c0 con 0x7f3a181092b0 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181041e0 0x7f3a181a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181041e0 0x7f3a181a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46098/0 (socket says 172.21.15.94:46098) 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 -- 172.21.15.94:0/510474509 learned_addr learned my addr 172.21.15.94:0/510474509 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:57.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1e800640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a1819a600 0x7f3a1819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1d400640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a181092b0 0x7f3a181a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 -- 172.21.15.94:0/510474509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a181092b0 msgr2=0x7f3a181a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a181092b0 0x7f3a181a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 -- 172.21.15.94:0/510474509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a1819a600 msgr2=0x7f3a1819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a1819a600 0x7f3a1819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.243+0000 7f3a1de00640 1 -- 172.21.15.94:0/510474509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1819b120 con 0x7f3a181041e0 2024-04-25T15:12:57.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.244+0000 7f3a1de00640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181041e0 0x7f3a181a0450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005ab0 tx=0x7f3a0c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:57.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.244+0000 7f3a12a00640 1 -- 172.21.15.94:0/510474509 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c03f070 con 0x7f3a181041e0 2024-04-25T15:12:57.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.244+0000 7f3a12a00640 1 -- 172.21.15.94:0/510474509 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0c031b50 con 0x7f3a181041e0 2024-04-25T15:12:57.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.244+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1819b3b0 con 0x7f3a181041e0 2024-04-25T15:12:57.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.244+0000 7f3a12a00640 1 -- 172.21.15.94:0/510474509 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c035030 con 0x7f3a181041e0 2024-04-25T15:12:57.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.244+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a181a75a0 con 0x7f3a181041e0 2024-04-25T15:12:57.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.245+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a18068c90 con 0x7f3a181041e0 2024-04-25T15:12:57.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.247+0000 7f3a12a00640 1 -- 172.21.15.94:0/510474509 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3a0c042070 con 0x7f3a181041e0 2024-04-25T15:12:57.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.249+0000 7f3a12a00640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f3a000a3d60 0x7f3a000a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.249+0000 7f3a12a00640 1 -- 172.21.15.94:0/510474509 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7f3a0c0f16a0 con 0x7f3a181041e0 2024-04-25T15:12:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.249+0000 7f3a1d400640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f3a000a3d60 0x7f3a000a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.250+0000 7f3a1d400640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f3a000a3d60 0x7f3a000a6220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3a1810a840 tx=0x7f3a0800b390 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:57.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.251+0000 7f3a12a00640 1 -- 172.21.15.94:0/510474509 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a0c0bd680 con 0x7f3a181041e0 2024-04-25T15:12:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:57 smithi138 ceph-mon[32875]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:57.457 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:57 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/1461451493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:12:57.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:57 smithi094 ceph-mon[27862]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:57.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:57 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/1461451493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:12:57.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.496+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a18104b90 con 0x7f3a181041e0 2024-04-25T15:12:57.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.498+0000 7f3a12a00640 1 -- 172.21.15.94:0/510474509 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11819 (secure 0 0 0) 0x7f3a0c0c1980 con 0x7f3a181041e0 2024-04-25T15:12:57.499 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:12:57.499 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14394,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/3392009193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":3392009193},{"type":"v1","addr":"172.21.15.94:6835","nonce":3392009193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14406,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/1161222965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":1161222965},{"type":"v1","addr":"172.21.15.94:6837","nonce":1161222965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/2559487009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":2559487009},{"type":"v1","addr":"172.21.15.94:6841","nonce":2559487009}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24454,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/2043270325","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2043270325},{"type":"v1","addr":"172.21.15.138:6833","nonce":2043270325}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24464,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6833/2696996187","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6832","nonce":2696996187},{"type":"v1","addr":"172.21.15.137:6833","nonce":2696996187}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24506,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6835/1346240831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6834","nonce":1346240831},{"type":"v1","addr":"172.21.15.137:6835","nonce":1346240831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/4151262699","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":4151262699},{"type":"v1","addr":"172.21.15.138:6835","nonce":4151262699}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24529,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/4267251536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":4267251536},{"type":"v1","addr":"172.21.15.138:6837","nonce":4267251536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24542,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.137:6837/4197829661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6836","nonce":4197829661},{"type":"v1","addr":"172.21.15.137:6837","nonce":4197829661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/593747621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":593747621},{"type":"v1","addr":"172.21.15.138:6839","nonce":593747621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T14:59:45.806095+0000","modified":"2024-04-25T14:59:50.088429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24566":{"gid":24566,"name":"g","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.94:6839/3586316280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":3586316280},{"type":"v1","addr":"172.21.15.94:6839","nonce":3586316280}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24596":{"gid":24596,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.137:6839/3444396688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6838","nonce":3444396688},{"type":"v1","addr":"172.21.15.137:6839","nonce":3444396688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24596,"qdb_cluster":[24596]},"id":1}]} 2024-04-25T15:12:57.499 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 19 2024-04-25T15:12:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f3a000a3d60 msgr2=0x7f3a000a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f3a000a3d60 0x7f3a000a6220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3a1810a840 tx=0x7f3a0800b390 comp rx=0 tx=0).stop 2024-04-25T15:12:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181041e0 msgr2=0x7f3a181a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181041e0 0x7f3a181a0450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c005ab0 tx=0x7f3a0c003c20 comp rx=0 tx=0).stop 2024-04-25T15:12:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 shutdown_connections 2024-04-25T15:12:57.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f3a000a3d60 0x7f3a000a6220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f3a1819a600 0x7f3a1819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a181092b0 0x7f3a181a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 --2- 172.21.15.94:0/510474509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3a181041e0 0x7f3a181a0450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 >> 172.21.15.94:0/510474509 conn(0x7f3a180ffa20 msgr2=0x7f3a18100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:57.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 shutdown_connections 2024-04-25T15:12:57.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.501+0000 7f3a1f200640 1 -- 172.21.15.94:0/510474509 wait complete. 2024-04-25T15:12:57.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:12:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 -- 172.21.15.94:0/377683389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c0109470 msgr2=0x7ff1c0111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/377683389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c0109470 0x7ff1c0111040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009ca0 tx=0x7ff1b802a8c0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 -- 172.21.15.94:0/377683389 shutdown_connections 2024-04-25T15:12:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/377683389 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c0109470 0x7ff1c0111040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/377683389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c0104d50 0x7ff1c0108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/377683389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c01043a0 0x7ff1c0104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 -- 172.21.15.94:0/377683389 >> 172.21.15.94:0/377683389 conn(0x7ff1c00ffc40 msgr2=0x7ff1c0102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.714+0000 7ff1c5e00640 1 -- 172.21.15.94:0/377683389 shutdown_connections 2024-04-25T15:12:57.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.715+0000 7ff1c5e00640 1 -- 172.21.15.94:0/377683389 wait complete. 2024-04-25T15:12:57.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.715+0000 7ff1c5e00640 1 Processor -- start 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1c5e00640 1 -- start start 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c01043a0 0x7ff1c01a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0109470 0x7ff1c01a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1c5e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c019a770 0x7ff1c019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1c5e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0114410 con 0x7ff1c01043a0 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1c5e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0114290 con 0x7ff1c019a770 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1c5e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0114590 con 0x7ff1c0109470 2024-04-25T15:12:57.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c01043a0 0x7ff1c01a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1bf400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c019a770 0x7ff1c019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1be000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0109470 0x7ff1c01a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c01043a0 0x7ff1c01a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46114/0 (socket says 172.21.15.94:46114) 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.716+0000 7ff1bea00640 1 -- 172.21.15.94:0/2217511318 learned_addr learned my addr 172.21.15.94:0/2217511318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1bea00640 1 -- 172.21.15.94:0/2217511318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0109470 msgr2=0x7ff1c01a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1bea00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0109470 0x7ff1c01a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1bea00640 1 -- 172.21.15.94:0/2217511318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c019a770 msgr2=0x7ff1c019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1bea00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c019a770 0x7ff1c019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1bea00640 1 -- 172.21.15.94:0/2217511318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c019b370 con 0x7ff1c01043a0 2024-04-25T15:12:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1bea00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c01043a0 0x7ff1c01a06b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e9f0 tx=0x7ff1b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1c4a00640 1 -- 172.21.15.94:0/2217511318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000cd80 con 0x7ff1c01043a0 2024-04-25T15:12:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1c4a00640 1 -- 172.21.15.94:0/2217511318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1b00108e0 con 0x7ff1c01043a0 2024-04-25T15:12:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c01a7450 con 0x7ff1c01043a0 2024-04-25T15:12:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1c4a00640 1 -- 172.21.15.94:0/2217511318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0018980 con 0x7ff1c01043a0 2024-04-25T15:12:57.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.717+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c01a7850 con 0x7ff1c01043a0 2024-04-25T15:12:57.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.718+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c0105870 con 0x7ff1c01043a0 2024-04-25T15:12:57.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.720+0000 7ff1c4a00640 1 -- 172.21.15.94:0/2217511318 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff1b0010430 con 0x7ff1c01043a0 2024-04-25T15:12:57.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.722+0000 7ff1c4a00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1a40a3d20 0x7ff1a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:57.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.722+0000 7ff1c4a00640 1 -- 172.21.15.94:0/2217511318 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7ff1b00d0b30 con 0x7ff1c01043a0 2024-04-25T15:12:57.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.722+0000 7ff1be000640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1a40a3d20 0x7ff1a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:57.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.723+0000 7ff1be000640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1a40a3d20 0x7ff1a40a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1c019bfc0 tx=0x7ff1ac005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:57.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.726+0000 7ff1c4a00640 1 -- 172.21.15.94:0/2217511318 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1b009cb10 con 0x7ff1c01043a0 2024-04-25T15:12:57.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:57 smithi137 ceph-mon[32439]: pgmap v334: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T15:12:57.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:57 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/1461451493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:12:57.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.929+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1c010c760 con 0x7ff1c01043a0 2024-04-25T15:12:57.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.934+0000 7ff1c4a00640 1 -- 172.21.15.94:0/2217511318 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+23151 (secure 0 0 0) 0x7ff1b00a0e10 con 0x7ff1c01043a0 2024-04-25T15:12:57.935 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-04-25T15:12:57.935 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":97,"fsid":"f04d929c-0312-11ef-bc93-c7b262605968","created":"2024-04-25T14:50:45.400368+0000","modified":"2024-04-25T15:12:17.235043+0000","last_up_change":"2024-04-25T14:58:51.366082+0000","last_in_change":"2024-04-25T14:58:31.651960+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T14:54:45.505952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T14:59:43.356930+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T14:59:44.767550+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a46fefd8-a368-4c2a-9038-54b8701077b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6803","nonce":1239034236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6805","nonce":1239034236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6809","nonce":1239034236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":1239034236},{"type":"v1","addr":"172.21.15.94:6807","nonce":1239034236}]},"public_addr":"172.21.15.94:6803/1239034236","cluster_addr":"172.21.15.94:6805/1239034236","heartbeat_back_addr":"172.21.15.94:6809/1239034236","heartbeat_front_addr":"172.21.15.94:6807/1239034236","state":["exists","up"]},{"osd":1,"uuid":"385d47c9-0cd9-424b-b924-411ae615e8ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6800","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6801","nonce":3278622901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6802","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6803","nonce":3278622901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6806","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6807","nonce":3278622901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6804","nonce":3278622901},{"type":"v1","addr":"172.21.15.137:6805","nonce":3278622901}]},"public_addr":"172.21.15.137:6801/3278622901","cluster_addr":"172.21.15.137:6803/3278622901","heartbeat_back_addr":"172.21.15.137:6807/3278622901","heartbeat_front_addr":"172.21.15.137:6805/3278622901","state":["exists","up"]},{"osd":2,"uuid":"3318c983-93b2-43b5-ad3b-a833ac35c895","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6801","nonce":3663095065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6803","nonce":3663095065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6807","nonce":3663095065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":3663095065},{"type":"v1","addr":"172.21.15.138:6805","nonce":3663095065}]},"public_addr":"172.21.15.138:6801/3663095065","cluster_addr":"172.21.15.138:6803/3663095065","heartbeat_back_addr":"172.21.15.138:6807/3663095065","heartbeat_front_addr":"172.21.15.138:6805/3663095065","state":["exists","up"]},{"osd":3,"uuid":"0afc1aca-e4d4-405f-a3e6-f80504727eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6811","nonce":462961415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6813","nonce":462961415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6817","nonce":462961415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":462961415},{"type":"v1","addr":"172.21.15.94:6815","nonce":462961415}]},"public_addr":"172.21.15.94:6811/462961415","cluster_addr":"172.21.15.94:6813/462961415","heartbeat_back_addr":"172.21.15.94:6817/462961415","heartbeat_front_addr":"172.21.15.94:6815/462961415","state":["exists","up"]},{"osd":4,"uuid":"a8898932-385f-4e12-a84d-074556811d16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6808","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6809","nonce":2944995107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6810","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6811","nonce":2944995107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6814","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6815","nonce":2944995107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6812","nonce":2944995107},{"type":"v1","addr":"172.21.15.137:6813","nonce":2944995107}]},"public_addr":"172.21.15.137:6809/2944995107","cluster_addr":"172.21.15.137:6811/2944995107","heartbeat_back_addr":"172.21.15.137:6815/2944995107","heartbeat_front_addr":"172.21.15.137:6813/2944995107","state":["exists","up"]},{"osd":5,"uuid":"79a31103-aabc-450e-8932-cd75be992a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6809","nonce":100945192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6811","nonce":100945192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6815","nonce":100945192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":100945192},{"type":"v1","addr":"172.21.15.138:6813","nonce":100945192}]},"public_addr":"172.21.15.138:6809/100945192","cluster_addr":"172.21.15.138:6811/100945192","heartbeat_back_addr":"172.21.15.138:6815/100945192","heartbeat_front_addr":"172.21.15.138:6813/100945192","state":["exists","up"]},{"osd":6,"uuid":"a27a3f85-5246-40dc-bbda-111d7e2f806b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6819","nonce":194684740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6821","nonce":194684740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6825","nonce":194684740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":194684740},{"type":"v1","addr":"172.21.15.94:6823","nonce":194684740}]},"public_addr":"172.21.15.94:6819/194684740","cluster_addr":"172.21.15.94:6821/194684740","heartbeat_back_addr":"172.21.15.94:6825/194684740","heartbeat_front_addr":"172.21.15.94:6823/194684740","state":["exists","up"]},{"osd":7,"uuid":"4efdc76d-0a2a-4b57-bfa4-b5961a0b90fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6816","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6817","nonce":2535537173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6818","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6819","nonce":2535537173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6822","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6823","nonce":2535537173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6820","nonce":2535537173},{"type":"v1","addr":"172.21.15.137:6821","nonce":2535537173}]},"public_addr":"172.21.15.137:6817/2535537173","cluster_addr":"172.21.15.137:6819/2535537173","heartbeat_back_addr":"172.21.15.137:6823/2535537173","heartbeat_front_addr":"172.21.15.137:6821/2535537173","state":["exists","up"]},{"osd":8,"uuid":"571dbd3c-a8f3-425f-a34c-f0f2c168907a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6817","nonce":3378906429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6819","nonce":3378906429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6823","nonce":3378906429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":3378906429},{"type":"v1","addr":"172.21.15.138:6821","nonce":3378906429}]},"public_addr":"172.21.15.138:6817/3378906429","cluster_addr":"172.21.15.138:6819/3378906429","heartbeat_back_addr":"172.21.15.138:6823/3378906429","heartbeat_front_addr":"172.21.15.138:6821/3378906429","state":["exists","up"]},{"osd":9,"uuid":"f0b7d2e9-2dd4-4dbf-befd-e860062b42ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6827","nonce":1308519263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6829","nonce":1308519263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6833","nonce":1308519263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1308519263},{"type":"v1","addr":"172.21.15.94:6831","nonce":1308519263}]},"public_addr":"172.21.15.94:6827/1308519263","cluster_addr":"172.21.15.94:6829/1308519263","heartbeat_back_addr":"172.21.15.94:6833/1308519263","heartbeat_front_addr":"172.21.15.94:6831/1308519263","state":["exists","up"]},{"osd":10,"uuid":"3a0daed0-84c8-4c9f-977e-d2502f39441e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6824","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6825","nonce":3255175900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6826","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6827","nonce":3255175900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6830","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6831","nonce":3255175900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.137:6828","nonce":3255175900},{"type":"v1","addr":"172.21.15.137:6829","nonce":3255175900}]},"public_addr":"172.21.15.137:6825/3255175900","cluster_addr":"172.21.15.137:6827/3255175900","heartbeat_back_addr":"172.21.15.137:6831/3255175900","heartbeat_front_addr":"172.21.15.137:6829/3255175900","state":["exists","up"]},{"osd":11,"uuid":"3b938f69-8356-40c2-9a76-45bed8d6a261","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6825","nonce":883275794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6827","nonce":883275794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6831","nonce":883275794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":883275794},{"type":"v1","addr":"172.21.15.138:6829","nonce":883275794}]},"public_addr":"172.21.15.138:6825/883275794","cluster_addr":"172.21.15.138:6827/883275794","heartbeat_back_addr":"172.21.15.138:6831/883275794","heartbeat_front_addr":"172.21.15.138:6829/883275794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:53:52.707724+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:17.977451+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:54:41.589648+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:13.329495+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:55:39.300866+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:03.792184+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:56:35.793737+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:01.955960+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:26.458498+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:57:58.546108+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:24.562776+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T14:58:49.499281+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/702744035":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3354212797":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6801/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1994260887":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/3440614830":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/1490569056":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6801/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:6800/1217751898":"2024-04-26T14:52:01.342497+0000","172.21.15.94:0/2523030980":"2024-04-26T15:02:09.415076+0000","172.21.15.94:6800/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6800/2485357518":"2024-04-26T15:02:09.415076+0000","172.21.15.94:0/1137535670":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/2519672579":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/312144263":"2024-04-26T14:51:27.240380+0000","172.21.15.94:6801/17546499":"2024-04-26T14:51:27.240380+0000","172.21.15.94:0/3116385060":"2024-04-26T14:52:01.342497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:12:57.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.936+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1a40a3d20 msgr2=0x7ff1a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.936+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1a40a3d20 0x7ff1a40a61e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1c019bfc0 tx=0x7ff1ac005f90 comp rx=0 tx=0).stop 2024-04-25T15:12:57.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.936+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c01043a0 msgr2=0x7ff1c01a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.936+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c01043a0 0x7ff1c01a06b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000e9f0 tx=0x7ff1b000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 shutdown_connections 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff1a40a3d20 0x7ff1a40a61e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff1c019a770 0x7ff1c019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1c0109470 0x7ff1c01a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 --2- 172.21.15.94:0/2217511318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1c01043a0 0x7ff1c01a06b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 >> 172.21.15.94:0/2217511318 conn(0x7ff1c00ffc40 msgr2=0x7ff1c0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:57.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 shutdown_connections 2024-04-25T15:12:57.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:57.937+0000 7ff1c5e00640 1 -- 172.21.15.94:0/2217511318 wait complete. 2024-04-25T15:12:57.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T15:12:58.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.167+0000 7ff602200640 1 -- 172.21.15.94:0/502885558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 msgr2=0x7ff5fc06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.167+0000 7ff602200640 1 --2- 172.21.15.94:0/502885558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc06ac20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009860 tx=0x7ff5ec02a340 comp rx=0 tx=0).stop 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.167+0000 7ff602200640 1 -- 172.21.15.94:0/502885558 shutdown_connections 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.167+0000 7ff602200640 1 --2- 172.21.15.94:0/502885558 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5fc107270 0x7ff5fc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.168+0000 7ff602200640 1 --2- 172.21.15.94:0/502885558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5fc06b1f0 0x7ff5fc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.168+0000 7ff602200640 1 --2- 172.21.15.94:0/502885558 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc06ac20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.168+0000 7ff602200640 1 -- 172.21.15.94:0/502885558 >> 172.21.15.94:0/502885558 conn(0x7ff5fc077eb0 msgr2=0x7ff5fc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.168+0000 7ff602200640 1 -- 172.21.15.94:0/502885558 shutdown_connections 2024-04-25T15:12:58.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.168+0000 7ff602200640 1 -- 172.21.15.94:0/502885558 wait complete. 2024-04-25T15:12:58.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.169+0000 7ff602200640 1 Processor -- start 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.169+0000 7ff602200640 1 -- start start 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.169+0000 7ff602200640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.169+0000 7ff602200640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5fc107270 0x7ff5fc198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.169+0000 7ff602200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5fc19cfe0 0x7ff5fc19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.169+0000 7ff602200640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc1121b0 con 0x7ff5fc107270 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.169+0000 7ff602200640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc112030 con 0x7ff5fc06a840 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff602200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc112330 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5faa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5fc107270 0x7ff5fc198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5fc19cfe0 0x7ff5fc19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fb400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.137:3300/0 says I am v2:172.21.15.94:53162/0 (socket says 172.21.15.94:53162) 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fb400640 1 -- 172.21.15.94:0/4255944243 learned_addr learned my addr 172.21.15.94:0/4255944243 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fbe00640 1 -- 172.21.15.94:0/4255944243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 msgr2=0x7ff5fc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fbe00640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fbe00640 1 -- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5fc107270 msgr2=0x7ff5fc198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fbe00640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5fc107270 0x7ff5fc198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fbe00640 1 -- 172.21.15.94:0/4255944243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19dad0 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.170+0000 7ff5fb400640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:12:58.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.171+0000 7ff5fbe00640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5fc19cfe0 0x7ff5fc19d490 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009fd0 tx=0x7ff5f400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:58.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.171+0000 7ff5f3e00640 1 -- 172.21.15.94:0/4255944243 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400ce20 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.171+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19ddc0 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.171+0000 7ff5f3e00640 1 -- 172.21.15.94:0/4255944243 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5f4005020 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.171+0000 7ff5f3e00640 1 -- 172.21.15.94:0/4255944243 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4010690 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.171+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a55b0 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.172+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc0691c0 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.175+0000 7ff5f3e00640 1 -- 172.21.15.94:0/4255944243 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff5f40054d0 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.176+0000 7ff5f3e00640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff5e00a3da0 0x7ff5e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.177+0000 7ff5f3e00640 1 -- 172.21.15.94:0/4255944243 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8542+0+0 (secure 0 0 0) 0x7ff5f40d0db0 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.177+0000 7ff5fb400640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff5e00a3da0 0x7ff5e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.177+0000 7ff5fb400640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff5e00a3da0 0x7ff5e00a6260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0095f0 tx=0x7ff5ec030340 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:58.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.178+0000 7ff5f3e00640 1 -- 172.21.15.94:0/4255944243 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5f409d440 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.420+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff5fc199a70 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.430+0000 7ff5f3e00640 1 -- 172.21.15.94:0/4255944243 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7ff5fc199a70 con 0x7ff5fc19cfe0 2024-04-25T15:12:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T15:12:58.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff5e00a3da0 msgr2=0x7ff5e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff5e00a3da0 0x7ff5e00a6260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0095f0 tx=0x7ff5ec030340 comp rx=0 tx=0).stop 2024-04-25T15:12:58.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5fc19cfe0 msgr2=0x7ff5fc19d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5fc19cfe0 0x7ff5fc19d490 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009fd0 tx=0x7ff5f400ef10 comp rx=0 tx=0).stop 2024-04-25T15:12:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 shutdown_connections 2024-04-25T15:12:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7ff5e00a3da0 0x7ff5e00a6260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff5fc19cfe0 0x7ff5fc19d490 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff5fc107270 0x7ff5fc198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 --2- 172.21.15.94:0/4255944243 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7ff5fc06a840 0x7ff5fc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.432+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 >> 172.21.15.94:0/4255944243 conn(0x7ff5fc077eb0 msgr2=0x7ff5fc108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.433+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 shutdown_connections 2024-04-25T15:12:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.433+0000 7ff602200640 1 -- 172.21.15.94:0/4255944243 wait complete. 2024-04-25T15:12:58.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T15:12:58.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:58 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/510474509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:12:58.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:58 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/2217511318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:58.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a[27858]: 2024-04-25T15:12:58.422+0000 7fc9cae00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:12:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 -- 172.21.15.94:0/2123667726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc104400 msgr2=0x7f88fc1047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 --2- 172.21.15.94:0/2123667726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc104400 0x7f88fc1047e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f88ec009900 tx=0x7f88ec02a400 comp rx=0 tx=0).stop 2024-04-25T15:12:58.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 -- 172.21.15.94:0/2123667726 shutdown_connections 2024-04-25T15:12:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 --2- 172.21.15.94:0/2123667726 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88fc1094d0 0x7f88fc1110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 --2- 172.21.15.94:0/2123667726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc104db0 0x7f88fc108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 --2- 172.21.15.94:0/2123667726 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc104400 0x7f88fc1047e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 -- 172.21.15.94:0/2123667726 >> 172.21.15.94:0/2123667726 conn(0x7f88fc0ffce0 msgr2=0x7f88fc102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 -- 172.21.15.94:0/2123667726 shutdown_connections 2024-04-25T15:12:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.628+0000 7f8902800640 1 -- 172.21.15.94:0/2123667726 wait complete. 2024-04-25T15:12:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.629+0000 7f8902800640 1 Processor -- start 2024-04-25T15:12:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.629+0000 7f8902800640 1 -- start start 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f8902800640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88fc104400 0x7f88fc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f8902800640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc1094d0 0x7f88fc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f8902800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc19a7f0 0x7f88fc19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f8902800640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc114490 con 0x7f88fc1094d0 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f8902800640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc114310 con 0x7f88fc104400 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f8902800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f88fc114610 con 0x7f88fc19a7f0 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc1094d0 0x7f88fc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc1094d0 0x7f88fc1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46154/0 (socket says 172.21.15.94:46154) 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 -- 172.21.15.94:0/3877907954 learned_addr learned my addr 172.21.15.94:0/3877907954 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88fb400640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88fc104400 0x7f88fc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88fbe00640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc19a7f0 0x7f88fc19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 -- 172.21.15.94:0/3877907954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc19a7f0 msgr2=0x7f88fc19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc19a7f0 0x7f88fc19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 -- 172.21.15.94:0/3877907954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88fc104400 msgr2=0x7f88fc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88fc104400 0x7f88fc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:58.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88faa00640 1 -- 172.21.15.94:0/3877907954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88fc19b380 con 0x7f88fc1094d0 2024-04-25T15:12:58.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.630+0000 7f88fbe00640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc19a7f0 0x7f88fc19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:12:58.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.631+0000 7f88faa00640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc1094d0 0x7f88fc1a0bc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f88e800e9c0 tx=0x7f88e800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:58.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.631+0000 7f88f3e00640 1 -- 172.21.15.94:0/3877907954 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e800cd80 con 0x7f88fc1094d0 2024-04-25T15:12:58.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.631+0000 7f88f3e00640 1 -- 172.21.15.94:0/3877907954 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e80054d0 con 0x7f88fc1094d0 2024-04-25T15:12:58.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.631+0000 7f88f3e00640 1 -- 172.21.15.94:0/3877907954 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e8010430 con 0x7f88fc1094d0 2024-04-25T15:12:58.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.631+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88fc19b670 con 0x7f88fc1094d0 2024-04-25T15:12:58.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.631+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88fc1a7800 con 0x7f88fc1094d0 2024-04-25T15:12:58.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.633+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc1058d0 con 0x7f88fc1094d0 2024-04-25T15:12:58.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.636+0000 7f88f3e00640 1 -- 172.21.15.94:0/3877907954 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88e8004fb0 con 0x7f88fc1094d0 2024-04-25T15:12:58.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.637+0000 7f88f3e00640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f88e00a3da0 0x7f88e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:58.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.637+0000 7f88f3e00640 1 -- 172.21.15.94:0/3877907954 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8584+0+0 (secure 0 0 0) 0x7f88e80d0e60 con 0x7f88fc1094d0 2024-04-25T15:12:58.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.638+0000 7f88fb400640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f88e00a3da0 0x7f88e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:58.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.638+0000 7f88fb400640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f88e00a3da0 0x7f88e00a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88ec005ab0 tx=0x7f88ec005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:58.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.641+0000 7f88f3e00640 1 -- 172.21.15.94:0/3877907954 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88e8053ff0 con 0x7f88fc1094d0 2024-04-25T15:12:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:58 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/510474509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:12:58.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:58 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/2217511318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:58.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:58 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/510474509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:12:58.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:58 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/2217511318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:12:58.733 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:12:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b[63292]: ignoring --setuser ceph since I am not root 2024-04-25T15:12:58.733 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:12:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b[63292]: ignoring --setgroup ceph since I am not root 2024-04-25T15:12:58.733 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:12:58 smithi137 ceph-mds[63296]: -- 172.21.15.137:0/913569176 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557bf7c89a40 con 0x557bf7c5cc00 2024-04-25T15:12:58.734 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:12:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b[63292]: starting mds.b at 2024-04-25T15:12:58.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:58.887+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f88fc104db0 con 0x7f88fc1094d0 2024-04-25T15:12:58.963 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:12:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g[69135]: ignoring --setuser ceph since I am not root 2024-04-25T15:12:58.963 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:12:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g[69135]: ignoring --setgroup ceph since I am not root 2024-04-25T15:12:58.963 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:12:58 smithi094 ceph-mds[69140]: -- 172.21.15.94:0/516769164 <== mon.2 v2:172.21.15.137:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5643ca917a40 con 0x5643ca8eac00 2024-04-25T15:12:58.963 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:12:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g[69135]: starting mds.g at 2024-04-25T15:12:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.428+0000 7f88f3e00640 1 -- 172.21.15.94:0/3877907954 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f88fc19b670 con 0x7f88fc1094d0 2024-04-25T15:12:59.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f88e00a3da0 msgr2=0x7f88e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:59.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f88e00a3da0 0x7f88e00a6260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88ec005ab0 tx=0x7f88ec005a20 comp rx=0 tx=0).stop 2024-04-25T15:12:59.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc1094d0 msgr2=0x7f88fc1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:59.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc1094d0 0x7f88fc1a0bc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f88e800e9c0 tx=0x7f88e800ee80 comp rx=0 tx=0).stop 2024-04-25T15:12:59.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 shutdown_connections 2024-04-25T15:12:59.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f88e00a3da0 0x7f88e00a6260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88fc19a7f0 0x7f88fc19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88fc1094d0 0x7f88fc1a0bc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 --2- 172.21.15.94:0/3877907954 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f88fc104400 0x7f88fc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.431+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 >> 172.21.15.94:0/3877907954 conn(0x7f88fc0ffce0 msgr2=0x7f88fc1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:59.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.432+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 shutdown_connections 2024-04-25T15:12:59.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.432+0000 7f8902800640 1 -- 172.21.15.94:0/3877907954 wait complete. 2024-04-25T15:12:59.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T15:12:59.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:59.549 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/4255944243' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:12:59.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:12:59.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:12:59.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:12:59.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T15:12:59.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T15:12:59.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T15:12:59.550 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:12:59 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3877907954' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:12:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 -- 172.21.15.94:0/3943293046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103010 msgr2=0x7fe6c01033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 --2- 172.21.15.94:0/3943293046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103010 0x7fe6c01033f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009900 tx=0x7fe6b402a480 comp rx=0 tx=0).stop 2024-04-25T15:12:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 -- 172.21.15.94:0/3943293046 shutdown_connections 2024-04-25T15:12:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 --2- 172.21.15.94:0/3943293046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6c010c430 0x7fe6c010e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 --2- 172.21.15.94:0/3943293046 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c0103930 0x7fe6c010be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 --2- 172.21.15.94:0/3943293046 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103010 0x7fe6c01033f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 -- 172.21.15.94:0/3943293046 >> 172.21.15.94:0/3943293046 conn(0x7fe6c00fc870 msgr2=0x7fe6c00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:12:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 -- 172.21.15.94:0/3943293046 shutdown_connections 2024-04-25T15:12:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.683+0000 7fe6c6400640 1 -- 172.21.15.94:0/3943293046 wait complete. 2024-04-25T15:12:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.684+0000 7fe6c6400640 1 Processor -- start 2024-04-25T15:12:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.684+0000 7fe6c6400640 1 -- start start 2024-04-25T15:12:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.684+0000 7fe6c6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6c0103010 0x7fe6c01a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c6400640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103930 0x7fe6c01a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c6400640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c010c430 0x7fe6c019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c6400640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0112110 con 0x7fe6c0103930 2024-04-25T15:12:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c6400640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0111f90 con 0x7fe6c010c430 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c6400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0112290 con 0x7fe6c0103010 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103930 0x7fe6c01a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103930 0x7fe6c01a0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46168/0 (socket says 172.21.15.94:46168) 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 -- 172.21.15.94:0/998580154 learned_addr learned my addr 172.21.15.94:0/998580154 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c5000640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6c0103010 0x7fe6c01a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c5a00640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c010c430 0x7fe6c019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 -- 172.21.15.94:0/998580154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6c0103010 msgr2=0x7fe6c01a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6c0103010 0x7fe6c01a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 -- 172.21.15.94:0/998580154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c010c430 msgr2=0x7fe6c019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c010c430 0x7fe6c019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6bfe00640 1 -- 172.21.15.94:0/998580154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c019afb0 con 0x7fe6c0103930 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.685+0000 7fe6c5a00640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c010c430 0x7fe6c019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.686+0000 7fe6bfe00640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103930 0x7fe6c01a0b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000e9f0 tx=0x7fe6b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.686+0000 7fe6bd600640 1 -- 172.21.15.94:0/998580154 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000ccc0 con 0x7fe6c0103930 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.686+0000 7fe6bd600640 1 -- 172.21.15.94:0/998580154 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6b00108e0 con 0x7fe6c0103930 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.686+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c019b2a0 con 0x7fe6c0103930 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.686+0000 7fe6bd600640 1 -- 172.21.15.94:0/998580154 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0018880 con 0x7fe6c0103930 2024-04-25T15:12:59.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.686+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a7560 con 0x7fe6c0103930 2024-04-25T15:12:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.687+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c0069d40 con 0x7fe6c0103930 2024-04-25T15:12:59.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.689+0000 7fe6bd600640 1 -- 172.21.15.94:0/998580154 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6b0010430 con 0x7fe6c0103930 2024-04-25T15:12:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.691+0000 7fe6bd600640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fe6a80a3da0 0x7fe6a80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:12:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.691+0000 7fe6bd600640 1 -- 172.21.15.94:0/998580154 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8584+0+0 (secure 0 0 0) 0x7fe6b00d0b70 con 0x7fe6c0103930 2024-04-25T15:12:59.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.691+0000 7fe6c5000640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fe6a80a3da0 0x7fe6a80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:12:59.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.692+0000 7fe6c5000640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fe6a80a3da0 0x7fe6a80a6260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005ab0 tx=0x7fe6b4005a20 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:12:59.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.695+0000 7fe6bd600640 1 -- 172.21.15.94:0/998580154 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6b009d170 con 0x7fe6c0103930 2024-04-25T15:12:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/4255944243' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:12:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:12:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:12:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:12:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T15:12:59.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T15:12:59.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T15:12:59.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:12:59 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3877907954' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:12:59.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: pgmap v335: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-25T15:12:59.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/4255944243' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:12:59.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T15:12:59.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:12:59.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:12:59.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T15:12:59.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T15:12:59.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-25T15:12:59.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:12:59 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3877907954' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:12:59.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:12:59.897+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe6c019bbd0 con 0x7fe6c0103930 2024-04-25T15:13:00.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.438+0000 7fe6bd600640 1 -- 172.21.15.94:0/998580154 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) ==== 181+0+0 (secure 0 0 0) 0x7fe6b009ca70 con 0x7fe6c0103930 2024-04-25T15:13:00.439 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-04-25T15:13:00.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.443+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fe6a80a3da0 msgr2=0x7fe6a80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:13:00.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.443+0000 7fe6c6400640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fe6a80a3da0 0x7fe6a80a6260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4005ab0 tx=0x7fe6b4005a20 comp rx=0 tx=0).stop 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.443+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103930 msgr2=0x7fe6c01a0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.443+0000 7fe6c6400640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103930 0x7fe6c01a0b20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6b000e9f0 tx=0x7fe6b000eeb0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 shutdown_connections 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7fe6a80a3da0 0x7fe6a80a6260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7fe6c010c430 0x7fe6c019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6c0103930 0x7fe6c01a0b20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 --2- 172.21.15.94:0/998580154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6c0103010 0x7fe6c01a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 >> 172.21.15.94:0/998580154 conn(0x7fe6c00fc870 msgr2=0x7fe6c0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:13:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 shutdown_connections 2024-04-25T15:13:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.444+0000 7fe6c6400640 1 -- 172.21.15.94:0/998580154 wait complete. 2024-04-25T15:13:00.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T15:13:00.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 -- 172.21.15.94:0/1876065013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04107130 msgr2=0x7f2c0410ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:13:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 --2- 172.21.15.94:0/1876065013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04107130 0x7f2c0410ed00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009ca0 tx=0x7f2bfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 -- 172.21.15.94:0/1876065013 shutdown_connections 2024-04-25T15:13:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 --2- 172.21.15.94:0/1876065013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04107130 0x7f2c0410ed00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 --2- 172.21.15.94:0/1876065013 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04102a10 0x7f2c04106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 --2- 172.21.15.94:0/1876065013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c04102060 0x7f2c04102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 -- 172.21.15.94:0/1876065013 >> 172.21.15.94:0/1876065013 conn(0x7f2c040fd920 msgr2=0x7f2c040ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:13:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 -- 172.21.15.94:0/1876065013 shutdown_connections 2024-04-25T15:13:00.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.665+0000 7f2c08e00640 1 -- 172.21.15.94:0/1876065013 wait complete. 2024-04-25T15:13:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 Processor -- start 2024-04-25T15:13:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 -- start start 2024-04-25T15:13:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04102060 0x7f2c041a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:13:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 --2- >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04107130 0x7f2c041a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:13:00.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0419a630 0x7f2c0419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:13:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 -- 0x7f2c041120d0 con 0x7f2c04102060 2024-04-25T15:13:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 -- --> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04111f50 con 0x7f2c04107130 2024-04-25T15:13:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.666+0000 7f2c08e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04112250 con 0x7f2c0419a630 2024-04-25T15:13:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0419a630 0x7f2c0419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:13:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0419a630 0x7f2c0419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.94:43202/0 (socket says 172.21.15.94:43202) 2024-04-25T15:13:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 -- 172.21.15.94:0/3981355157 learned_addr learned my addr 172.21.15.94:0/3981355157 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-04-25T15:13:00.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03400640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04102060 0x7f2c041a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:13:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c02a00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04107130 0x7f2c041a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:13:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 -- 172.21.15.94:0/3981355157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04107130 msgr2=0x7f2c041a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:13:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04107130 0x7f2c041a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 -- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04102060 msgr2=0x7f2c041a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:13:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04102060 0x7f2c041a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 -- 172.21.15.94:0/3981355157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0419b230 con 0x7f2c0419a630 2024-04-25T15:13:00.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.667+0000 7f2c03e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0419a630 0x7f2c0419aab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc02f540 tx=0x7f2bfc0316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:13:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.668+0000 7f2bfbe00640 1 -- 172.21.15.94:0/3981355157 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc003b70 con 0x7f2c0419a630 2024-04-25T15:13:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.668+0000 7f2bfbe00640 1 -- 172.21.15.94:0/3981355157 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bfc003d10 con 0x7f2c0419a630 2024-04-25T15:13:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.668+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0419b4c0 con 0x7f2c0419a630 2024-04-25T15:13:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.668+0000 7f2bfbe00640 1 -- 172.21.15.94:0/3981355157 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc02f990 con 0x7f2c0419a630 2024-04-25T15:13:00.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.668+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c041a7660 con 0x7f2c0419a630 2024-04-25T15:13:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.669+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c04103530 con 0x7f2c0419a630 2024-04-25T15:13:00.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.671+0000 7f2bfbe00640 1 -- 172.21.15.94:0/3981355157 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bfc004b40 con 0x7f2c0419a630 2024-04-25T15:13:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.672+0000 7f2bfbe00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2be80a3d20 0x7f2be80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:13:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.672+0000 7f2bfbe00640 1 -- 172.21.15.94:0/3981355157 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8088+0+0 (secure 0 0 0) 0x7f2bfc0f1a80 con 0x7f2c0419a630 2024-04-25T15:13:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.673+0000 7f2c03400640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2be80a3d20 0x7f2be80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:13:00.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.673+0000 7f2c03400640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2be80a3d20 0x7f2be80a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009bd0 tx=0x7f2bf4005f90 comp rx=0 tx=0).ready entity=mgr.15027 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:13:00.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.675+0000 7f2bfbe00640 1 -- 172.21.15.94:0/3981355157 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bfc0be390 con 0x7f2c0419a630 2024-04-25T15:13:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:13:00.707 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: Cluster is now healthy 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3877907954' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.137:6838/3819263418,v1:172.21.15.137:6839/3819263418] up:boot 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: mds.? [v2:172.21.15.94:6838/2332604164,v1:172.21.15.94:6839/2332604164] up:boot 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: fsmap 12 up:standby 2024-04-25T15:13:00.708 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:00 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/998580154' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:00.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:13:00.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:13:00.712 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: Cluster is now healthy 2024-04-25T15:13:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3877907954' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:13:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:13:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:13:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.137:6838/3819263418,v1:172.21.15.137:6839/3819263418] up:boot 2024-04-25T15:13:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: mds.? [v2:172.21.15.94:6838/2332604164,v1:172.21.15.94:6839/2332604164] up:boot 2024-04-25T15:13:00.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: fsmap 12 up:standby 2024-04-25T15:13:00.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/998580154' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:00.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:13:00.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:13:00.732 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: Cluster is now healthy 2024-04-25T15:13:00.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3877907954' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:13:00.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T15:13:00.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: from='mgr.15027 172.21.15.94:0/2435113509' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T15:13:00.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.137:6838/3819263418,v1:172.21.15.137:6839/3819263418] up:boot 2024-04-25T15:13:00.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: mds.? [v2:172.21.15.94:6838/2332604164,v1:172.21.15.94:6839/2332604164] up:boot 2024-04-25T15:13:00.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: fsmap 12 up:standby 2024-04-25T15:13:00.733 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:00 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/998580154' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:00.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:00.889+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2c04102ab0 con 0x7f2c0419a630 2024-04-25T15:13:01.212 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:13:00.884+0000 7ff5f0200700 -1 client.25168: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T15:13:01.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:13:00.887+0000 7ff5f0200700 -1 client.25168: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-04-25T15:13:01.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.447+0000 7f2bfbe00640 1 -- 172.21.15.94:0/3981355157 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) ==== 169+0+0 (secure 0 0 0) 0x7f2bfc0bdc90 con 0x7f2c0419a630 2024-04-25T15:13:01.448 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-04-25T15:13:01.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2be80a3d20 msgr2=0x7f2be80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:13:01.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2be80a3d20 0x7f2be80a61e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4009bd0 tx=0x7f2bf4005f90 comp rx=0 tx=0).stop 2024-04-25T15:13:01.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0419a630 msgr2=0x7f2c0419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:13:01.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0419a630 0x7f2c0419aab0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc02f540 tx=0x7f2bfc0316f0 comp rx=0 tx=0).stop 2024-04-25T15:13:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 shutdown_connections 2024-04-25T15:13:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:6800/2411855535,v1:172.21.15.94:6801/2411855535] conn(0x7f2be80a3d20 0x7f2be80a61e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c0419a630 0x7f2c0419aab0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.137:3300/0,v1:172.21.15.137:6789/0] conn(0x7f2c04107130 0x7f2c041a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 --2- 172.21.15.94:0/3981355157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2c04102060 0x7f2c041a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:13:01.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.454+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 >> 172.21.15.94:0/3981355157 conn(0x7f2c040fd920 msgr2=0x7f2c0410e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:13:01.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.455+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 shutdown_connections 2024-04-25T15:13:01.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-04-25T15:13:01.455+0000 7f2c08e00640 1 -- 172.21.15.94:0/3981355157 wait complete. 2024-04-25T15:13:01.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:13:01.547 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:13:01.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:01 smithi137 ceph-mon[32439]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:13:01.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:01 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/998580154' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:13:01.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:01 smithi137 ceph-mon[32439]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T15:13:01.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:01 smithi137 ceph-mon[32439]: from='client.? 172.21.15.94:0/3981355157' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:01.574 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:01 smithi137 ceph-mon[32439]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:01.579 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:13:01.605 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:01 smithi138 ceph-mon[32875]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:13:01.606 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:01 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/998580154' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:13:01.606 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:01 smithi138 ceph-mon[32875]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T15:13:01.606 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:01 smithi138 ceph-mon[32875]: from='client.? 172.21.15.94:0/3981355157' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:01.606 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:01 smithi138 ceph-mon[32875]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:01.610 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T15:13:01.610 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T15:13:01.667 DEBUG:teuthology.orchestra.run.smithi137:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T15:13:01.687 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T15:13:01.704 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T15:13:01.704 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T15:13:01.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a 2024-04-25T15:13:01.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:01 smithi094 ceph-mon[27862]: pgmap v337: 97 pgs: 97 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-25T15:13:01.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:01 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/998580154' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:13:01.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:01 smithi094 ceph-mon[27862]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T15:13:01.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:01 smithi094 ceph-mon[27862]: from='client.? 172.21.15.94:0/3981355157' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:01.714 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:01 smithi094 ceph-mon[27862]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:13:02.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:01 smithi094 systemd[1]: Stopping Ceph mon.a for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:02.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a[27858]: 2024-04-25T15:13:02.252+0000 7fc9d1c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:02.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a[27858]: 2024-04-25T15:13:02.252+0000 7fc9d1c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T15:13:02.713 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:02 smithi094 podman[85846]: 2024-04-25 15:13:02.482882403 +0000 UTC m=+0.372972370 container died f57cc4dcd35b57af7a01ab457ced21eec99dd2142f039a47d89cf2d8ea14093d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T15:13:03.096 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:02 smithi094 podman[85846]: 2024-04-25 15:13:02.87722836 +0000 UTC m=+0.767318348 container cleanup f57cc4dcd35b57af7a01ab457ced21eec99dd2142f039a47d89cf2d8ea14093d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T15:13:03.097 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:02 smithi094 bash[85846]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a 2024-04-25T15:13:03.462 INFO:journalctl@ceph.mon.a.smithi094.stdout:Apr 25 15:13:03 smithi094 podman[85860]: 2024-04-25 15:13:03.09513845 +0000 UTC m=+0.603888245 container remove f57cc4dcd35b57af7a01ab457ced21eec99dd2142f039a47d89cf2d8ea14093d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T15:13:03.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T15:13:03.740 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:03.741 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T15:13:03.741 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T15:13:03.741 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.b 2024-04-25T15:13:04.036 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:03 smithi137 systemd[1]: Stopping Ceph mon.b for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b[32435]: 2024-04-25T15:13:04.034+0000 7f4eaf44e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b[32435]: 2024-04-25T15:13:04.034+0000 7f4eaf44e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T15:13:04.483 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:04 smithi137 podman[68576]: 2024-04-25 15:13:04.239941097 +0000 UTC m=+0.303074336 container died af3d393348f445563be78afcc8bdc4348aa2dd6980c0927e3fdadf29488270b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T15:13:04.982 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:04 smithi137 podman[68576]: 2024-04-25 15:13:04.744621421 +0000 UTC m=+0.807754666 container cleanup af3d393348f445563be78afcc8bdc4348aa2dd6980c0927e3fdadf29488270b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T15:13:04.983 INFO:journalctl@ceph.mon.b.smithi137.stdout:Apr 25 15:13:04 smithi137 bash[68576]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-b 2024-04-25T15:13:05.176 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T15:13:05.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:05.219 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T15:13:05.219 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T15:13:05.220 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.c 2024-04-25T15:13:05.524 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:05 smithi138 systemd[1]: Stopping Ceph mon.c for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:05.916 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c[32857]: 2024-04-25T15:13:05.522+0000 7f6fecd47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:05.917 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c[32857]: 2024-04-25T15:13:05.522+0000 7f6fecd47700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T15:13:05.917 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:05 smithi138 podman[69283]: 2024-04-25 15:13:05.64635556 +0000 UTC m=+0.238223349 container died ef5fb5e28152dc3fb37c5d781771bcc5234d45846c1bcdcf0c3be3629882eaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T15:13:06.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:06 smithi138 podman[69283]: 2024-04-25 15:13:06.006481553 +0000 UTC m=+0.598349339 container cleanup ef5fb5e28152dc3fb37c5d781771bcc5234d45846c1bcdcf0c3be3629882eaa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T15:13:06.207 INFO:journalctl@ceph.mon.c.smithi138.stdout:Apr 25 15:13:06 smithi138 bash[69283]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mon-c 2024-04-25T15:13:06.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T15:13:06.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:06.420 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T15:13:06.421 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T15:13:06.421 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.x 2024-04-25T15:13:06.962 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:06 smithi094 systemd[1]: Stopping Ceph mgr.x for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:07.382 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:13:06.981+0000 7ff699600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:07.383 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x[28094]: 2024-04-25T15:13:06.981+0000 7ff699600700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T15:13:07.383 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:07 smithi094 podman[85970]: 2024-04-25 15:13:07.084707214 +0000 UTC m=+0.236736509 container died 2441008644f040a07eb7176ba01e877a9cd4dbd07c95fc46788dfa00d6ec6388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T15:13:07.712 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:07 smithi094 podman[85970]: 2024-04-25 15:13:07.548106922 +0000 UTC m=+0.700136268 container cleanup 2441008644f040a07eb7176ba01e877a9cd4dbd07c95fc46788dfa00d6ec6388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T15:13:07.713 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:07 smithi094 bash[85970]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x 2024-04-25T15:13:08.213 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Apr 25 15:13:07 smithi094 podman[85984]: 2024-04-25 15:13:07.825221341 +0000 UTC m=+0.723217364 container remove 2441008644f040a07eb7176ba01e877a9cd4dbd07c95fc46788dfa00d6ec6388 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T15:13:08.395 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T15:13:08.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:08.508 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T15:13:08.508 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T15:13:08.508 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.y 2024-04-25T15:13:08.854 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:13:08 smithi137 systemd[1]: Stopping Ceph mgr.y for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:09.117 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:13:08 smithi137 podman[68711]: 2024-04-25 15:13:08.853728726 +0000 UTC m=+0.158573629 container died 10124ef5b4b5fdbbd1965f84277725c3a6b3dbccfff43cd36b5ada70a3385f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T15:13:09.483 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:13:09 smithi137 podman[68711]: 2024-04-25 15:13:09.232425866 +0000 UTC m=+0.537270768 container cleanup 10124ef5b4b5fdbbd1965f84277725c3a6b3dbccfff43cd36b5ada70a3385f65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T15:13:09.483 INFO:journalctl@ceph.mgr.y.smithi137.stdout:Apr 25 15:13:09 smithi137 bash[68711]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-y 2024-04-25T15:13:09.655 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T15:13:09.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:09.691 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T15:13:09.691 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T15:13:09.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.z 2024-04-25T15:13:10.051 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:13:09 smithi138 systemd[1]: Stopping Ceph mgr.z for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:10.336 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:13:10 smithi138 podman[69405]: 2024-04-25 15:13:10.0502074 +0000 UTC m=+0.160306829 container died 2a448e3332f995b6ec4cc0ec710e5f0a7a92aa7a2a99c384bb0bf4459fdc6767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-25T15:13:10.604 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:13:10 smithi138 podman[69405]: 2024-04-25 15:13:10.335938768 +0000 UTC m=+0.446038197 container cleanup 2a448e3332f995b6ec4cc0ec710e5f0a7a92aa7a2a99c384bb0bf4459fdc6767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T15:13:10.604 INFO:journalctl@ceph.mgr.z.smithi138.stdout:Apr 25 15:13:10 smithi138 bash[69405]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mgr-z 2024-04-25T15:13:10.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T15:13:10.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:10.763 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T15:13:10.763 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T15:13:10.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.0 2024-04-25T15:13:11.212 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:10 smithi094 systemd[1]: Stopping Ceph osd.0 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:11.712 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0[42494]: 2024-04-25T15:13:11.334+0000 7f0a3b800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:11.713 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0[42494]: 2024-04-25T15:13:11.334+0000 7f0a3b800700 -1 osd.0 100 *** Got signal Terminated *** 2024-04-25T15:13:11.713 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0[42494]: 2024-04-25T15:13:11.334+0000 7f0a3b800700 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:13:17.121 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:16 smithi094 podman[86090]: 2024-04-25 15:13:16.856225448 +0000 UTC m=+5.679796343 container died 04e9af904d9da4004dd16ee0bb84f29dd468559fc17f5238c14b25bdd065f6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T15:13:17.462 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:17 smithi094 podman[86090]: 2024-04-25 15:13:17.303937872 +0000 UTC m=+6.127508800 container cleanup 04e9af904d9da4004dd16ee0bb84f29dd468559fc17f5238c14b25bdd065f6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:13:17.463 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:17 smithi094 bash[86090]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0 2024-04-25T15:13:17.962 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:17 smithi094 podman[86109]: 2024-04-25 15:13:17.580709772 +0000 UTC m=+0.706392205 container remove 04e9af904d9da4004dd16ee0bb84f29dd468559fc17f5238c14b25bdd065f6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T15:13:18.745 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:18 smithi094 podman[86204]: 2024-04-25 15:13:18.374366249 +0000 UTC m=+0.038179690 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:13:18.745 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:18 smithi094 podman[86204]: 2024-04-25 15:13:18.498146821 +0000 UTC m=+0.161960293 container create 9309054ee809e0892ad0aeba2a7926df6ce904bbbdd7af566e4a7a3ab5596869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T15:13:18.745 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:18 smithi094 podman[86204]: 2024-04-25 15:13:18.74389862 +0000 UTC m=+0.407712099 container init 9309054ee809e0892ad0aeba2a7926df6ce904bbbdd7af566e4a7a3ab5596869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T15:13:19.213 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:18 smithi094 podman[86204]: 2024-04-25 15:13:18.755629608 +0000 UTC m=+0.419443072 container start 9309054ee809e0892ad0aeba2a7926df6ce904bbbdd7af566e4a7a3ab5596869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T15:13:19.213 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:18 smithi094 podman[86204]: 2024-04-25 15:13:18.851098383 +0000 UTC m=+0.514911862 container attach 9309054ee809e0892ad0aeba2a7926df6ce904bbbdd7af566e4a7a3ab5596869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T15:13:19.560 INFO:journalctl@ceph.osd.0.smithi094.stdout:Apr 25 15:13:19 smithi094 podman[86204]: 2024-04-25 15:13:19.214233942 +0000 UTC m=+0.878047421 container died 9309054ee809e0892ad0aeba2a7926df6ce904bbbdd7af566e4a7a3ab5596869 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T15:13:19.900 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T15:13:19.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:19.985 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T15:13:19.985 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T15:13:19.985 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.1 2024-04-25T15:13:20.314 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:20 smithi137 systemd[1]: Stopping Ceph osd.1 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:20.732 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1[37053]: 2024-04-25T15:13:20.313+0000 7f4248149700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:20.732 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1[37053]: 2024-04-25T15:13:20.313+0000 7f4248149700 -1 osd.1 100 *** Got signal Terminated *** 2024-04-25T15:13:20.733 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1[37053]: 2024-04-25T15:13:20.313+0000 7f4248149700 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:13:26.142 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:25 smithi137 podman[68836]: 2024-04-25 15:13:25.859489734 +0000 UTC m=+5.684502155 container died 5b5c5406973c39752187de54976ecd53baf1ad43a96036ff5e15147b75b2edee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:13:26.482 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:26 smithi137 podman[68836]: 2024-04-25 15:13:26.223400188 +0000 UTC m=+6.048412609 container cleanup 5b5c5406973c39752187de54976ecd53baf1ad43a96036ff5e15147b75b2edee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T15:13:26.483 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:26 smithi137 bash[68836]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1 2024-04-25T15:13:26.776 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:26 smithi137 podman[68854]: 2024-04-25 15:13:26.483048004 +0000 UTC m=+0.626531980 container remove 5b5c5406973c39752187de54976ecd53baf1ad43a96036ff5e15147b75b2edee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T15:13:27.037 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:26 smithi137 podman[68945]: 2024-04-25 15:13:26.775643255 +0000 UTC m=+0.111425667 container create db30044da7a30fcca66e2f434a046a03aeb672115870aa7b2a81a818a228ef8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:13:27.037 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:26 smithi137 podman[68945]: 2024-04-25 15:13:26.682451461 +0000 UTC m=+0.018233888 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:13:27.037 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 podman[68945]: 2024-04-25 15:13:27.036533928 +0000 UTC m=+0.372316340 container init db30044da7a30fcca66e2f434a046a03aeb672115870aa7b2a81a818a228ef8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T15:13:27.335 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 podman[68945]: 2024-04-25 15:13:27.039771824 +0000 UTC m=+0.375554244 container start db30044da7a30fcca66e2f434a046a03aeb672115870aa7b2a81a818a228ef8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T15:13:27.335 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 podman[68945]: 2024-04-25 15:13:27.119798889 +0000 UTC m=+0.455581326 container attach db30044da7a30fcca66e2f434a046a03aeb672115870aa7b2a81a818a228ef8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-25T15:13:27.681 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 podman[68945]: 2024-04-25 15:13:27.334709439 +0000 UTC m=+0.670491854 container died db30044da7a30fcca66e2f434a046a03aeb672115870aa7b2a81a818a228ef8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:13:27.932 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 podman[68971]: 2024-04-25 15:13:27.914722033 +0000 UTC m=+0.574134092 container remove db30044da7a30fcca66e2f434a046a03aeb672115870aa7b2a81a818a228ef8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T15:13:27.932 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-25T15:13:27.932 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 systemd[1]: Stopped Ceph osd.1 for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T15:13:27.933 INFO:journalctl@ceph.osd.1.smithi137.stdout:Apr 25 15:13:27 smithi137 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.1.service: Consumed 9.844s CPU time. 2024-04-25T15:13:27.934 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T15:13:27.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:27.975 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T15:13:27.976 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T15:13:27.976 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.2 2024-04-25T15:13:28.273 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:28 smithi138 systemd[1]: Stopping Ceph osd.2 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:28.707 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T15:13:28.271+0000 7f80ddd61700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:28.707 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T15:13:28.271+0000 7f80ddd61700 -1 osd.2 100 *** Got signal Terminated *** 2024-04-25T15:13:28.707 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2[38026]: 2024-04-25T15:13:28.271+0000 7f80ddd61700 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:13:33.850 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:33 smithi138 podman[69541]: 2024-04-25 15:13:33.598197666 +0000 UTC m=+5.427566201 container died d403b70cd4e0418d475e34812cdf7a16d2ba2c44d7de8954be06a4e372f0ec92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:13:33.850 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:33 smithi138 podman[69541]: 2024-04-25 15:13:33.847184455 +0000 UTC m=+5.676553032 container cleanup d403b70cd4e0418d475e34812cdf7a16d2ba2c44d7de8954be06a4e372f0ec92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T15:13:33.850 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:33 smithi138 bash[69541]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2 2024-04-25T15:13:34.457 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:34 smithi138 podman[69559]: 2024-04-25 15:13:34.114863102 +0000 UTC m=+0.519722802 container remove d403b70cd4e0418d475e34812cdf7a16d2ba2c44d7de8954be06a4e372f0ec92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T15:13:34.727 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:34 smithi138 podman[69652]: 2024-04-25 15:13:34.364252068 +0000 UTC m=+0.017650845 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:13:34.728 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:34 smithi138 podman[69652]: 2024-04-25 15:13:34.49066671 +0000 UTC m=+0.144065447 container create bb2f693a006b438bd9614646e70491364fab37fa82c1848dfffad36e26f5795c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:13:35.031 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:34 smithi138 podman[69652]: 2024-04-25 15:13:34.727078236 +0000 UTC m=+0.380476992 container init bb2f693a006b438bd9614646e70491364fab37fa82c1848dfffad36e26f5795c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T15:13:35.032 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:34 smithi138 podman[69652]: 2024-04-25 15:13:34.730652293 +0000 UTC m=+0.384051030 container start bb2f693a006b438bd9614646e70491364fab37fa82c1848dfffad36e26f5795c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-25T15:13:35.032 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:34 smithi138 podman[69652]: 2024-04-25 15:13:34.801578216 +0000 UTC m=+0.454976954 container attach bb2f693a006b438bd9614646e70491364fab37fa82c1848dfffad36e26f5795c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:13:35.032 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:35 smithi138 podman[69652]: 2024-04-25 15:13:35.031115141 +0000 UTC m=+0.684513877 container died bb2f693a006b438bd9614646e70491364fab37fa82c1848dfffad36e26f5795c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T15:13:35.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T15:13:35.562 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:35 smithi138 podman[69677]: 2024-04-25 15:13:35.522061422 +0000 UTC m=+0.485264439 container remove bb2f693a006b438bd9614646e70491364fab37fa82c1848dfffad36e26f5795c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T15:13:35.562 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:35 smithi138 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-25T15:13:35.562 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:35 smithi138 systemd[1]: Stopped Ceph osd.2 for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T15:13:35.562 INFO:journalctl@ceph.osd.2.smithi138.stdout:Apr 25 15:13:35 smithi138 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.2.service: Consumed 9.846s CPU time. 2024-04-25T15:13:35.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:35.578 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T15:13:35.578 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T15:13:35.578 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.3 2024-04-25T15:13:35.963 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:35 smithi094 systemd[1]: Stopping Ceph osd.3 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:36.463 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:36 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3[49429]: 2024-04-25T15:13:36.105+0000 7fbd6a000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:36.464 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:36 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3[49429]: 2024-04-25T15:13:36.105+0000 7fbd6a000700 -1 osd.3 100 *** Got signal Terminated *** 2024-04-25T15:13:36.464 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:36 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3[49429]: 2024-04-25T15:13:36.105+0000 7fbd6a000700 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:13:37.962 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:37 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:37.674+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:38.962 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:38 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:38.649+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:39.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:39 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:39.447+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:39.962 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:39 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:39.646+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:40.462 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:39 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3[49429]: 2024-04-25T15:13:39.965+0000 7fbd65800700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.646092+0000 front 2024-04-25T15:13:15.646455+0000 (oldest deadline 2024-04-25T15:13:39.146106+0000) 2024-04-25T15:13:40.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:40.402+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:40.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:40 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:40.432+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:40.939 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:40 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:40.687+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:41.212 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:40 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3[49429]: 2024-04-25T15:13:40.937+0000 7fbd65800700 -1 osd.3 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.646092+0000 front 2024-04-25T15:13:15.646455+0000 (oldest deadline 2024-04-25T15:13:39.146106+0000) 2024-04-25T15:13:41.393 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:40.960+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:41.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:41.391+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:41.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:41 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:41.427+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:41.824 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:41 smithi094 podman[86322]: 2024-04-25 15:13:41.514260354 +0000 UTC m=+5.539844855 container died ea9503ecc2ed41829d1a92c68f41bbf5e4c707ae41bb25ae1b3f66442817925b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:13:41.824 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:41 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:41.735+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:42.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:41.938+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:42.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:41 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:41.838+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:42.213 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:41 smithi094 podman[86322]: 2024-04-25 15:13:41.967653336 +0000 UTC m=+5.993237834 container cleanup ea9503ecc2ed41829d1a92c68f41bbf5e4c707ae41bb25ae1b3f66442817925b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T15:13:42.213 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:41 smithi094 bash[86322]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3 2024-04-25T15:13:42.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:42.377+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:42.712 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:42 smithi094 podman[86340]: 2024-04-25 15:13:42.238115345 +0000 UTC m=+0.706467650 container remove ea9503ecc2ed41829d1a92c68f41bbf5e4c707ae41bb25ae1b3f66442817925b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T15:13:42.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:42 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:42.452+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:43.042 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:42 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:42.843+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:43.042 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:42 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:42.725+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:43.317 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:43 smithi094 podman[86426]: 2024-04-25 15:13:42.945710325 +0000 UTC m=+0.032930618 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:13:43.317 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:43 smithi094 podman[86426]: 2024-04-25 15:13:43.061891598 +0000 UTC m=+0.149111951 container create 484502ebe67245cf375afbb7a67cecd128467f273ea937e2fd8ac47be52c5113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T15:13:43.413 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:42.974+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:43.574 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:43 smithi094 podman[86426]: 2024-04-25 15:13:43.315334773 +0000 UTC m=+0.402555109 container init 484502ebe67245cf375afbb7a67cecd128467f273ea937e2fd8ac47be52c5113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T15:13:43.575 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:43 smithi094 podman[86426]: 2024-04-25 15:13:43.325007875 +0000 UTC m=+0.412228177 container start 484502ebe67245cf375afbb7a67cecd128467f273ea937e2fd8ac47be52c5113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T15:13:43.575 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:43 smithi094 podman[86426]: 2024-04-25 15:13:43.397733273 +0000 UTC m=+0.484953572 container attach 484502ebe67245cf375afbb7a67cecd128467f273ea937e2fd8ac47be52c5113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T15:13:43.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:43.412+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:43.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:43 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:43.466+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:43.846 INFO:journalctl@ceph.osd.3.smithi094.stdout:Apr 25 15:13:43 smithi094 podman[86426]: 2024-04-25 15:13:43.795434453 +0000 UTC m=+0.882654748 container died 484502ebe67245cf375afbb7a67cecd128467f273ea937e2fd8ac47be52c5113 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-25T15:13:43.847 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:43 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:43.766+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:44.134 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:43 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:43.844+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:44.393 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:43.962+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:44.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T15:13:44.630 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:44.631 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T15:13:44.631 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T15:13:44.631 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.4 2024-04-25T15:13:44.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:44.392+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:44.732 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:44 smithi137 systemd[1]: Stopping Ceph osd.4 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:44.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:44 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:44.448+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:45.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:44 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:44.785+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:45.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:44 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:44.878+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:45.232 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:44 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T15:13:44.950+0000 7f27a772f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:45.233 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:44 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T15:13:44.950+0000 7f27a772f700 -1 osd.4 100 *** Got signal Terminated *** 2024-04-25T15:13:45.233 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:44 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T15:13:44.950+0000 7f27a772f700 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:13:45.420 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:44.961+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:45.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:45.419+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:45.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:45 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:45.443+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:46.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:45.934+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:46.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:45 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:45.786+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:46.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:45 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:45.906+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:46.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:46.457+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:46.732 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:46 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T15:13:46.467+0000 7f279fd1b700 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.180550+0000 front 2024-04-25T15:13:22.180591+0000 (oldest deadline 2024-04-25T15:13:46.280728+0000) 2024-04-25T15:13:46.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:46 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:46.403+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:47.114 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:46 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:46.803+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:47.115 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:46 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:46.875+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:47.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:46.904+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:47.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:46.904+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:47.732 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:47 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T15:13:47.472+0000 7f279fd1b700 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.180550+0000 front 2024-04-25T15:13:22.180591+0000 (oldest deadline 2024-04-25T15:13:46.280728+0000) 2024-04-25T15:13:47.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:47 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:47.405+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:47.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:47 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:47.405+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:47.925 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:47.480+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:48.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:47.923+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:48.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:47.923+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:48.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:47 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:47.824+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:48.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:47 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:47.920+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:48.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:47 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:47.920+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:48.732 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:48 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T15:13:48.429+0000 7f279fd1b700 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.180550+0000 front 2024-04-25T15:13:22.180591+0000 (oldest deadline 2024-04-25T15:13:46.280728+0000) 2024-04-25T15:13:48.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:48 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:48.369+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:48.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:48 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:48.369+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:48.899 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:48 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:48.525+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:48.899 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:48 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:48.525+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:49.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:48 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:48.897+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:49.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:48 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:48.897+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:49.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:48 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:48.847+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:49.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:48 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:48.847+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:49.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:48 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:48.929+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:49.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:48 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:48.929+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:49.732 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:49 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4[44000]: 2024-04-25T15:13:49.421+0000 7f279fd1b700 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.180550+0000 front 2024-04-25T15:13:22.180591+0000 (oldest deadline 2024-04-25T15:13:46.280728+0000) 2024-04-25T15:13:49.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:49 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:49.358+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:49.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:49 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:49.358+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:49.885 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:49 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:49.478+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:49.885 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:49 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:49.478+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:50.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:49 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:49.883+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:50.207 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:49 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:49.883+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:50.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:50 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:50.114+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:50.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:49 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:49.867+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:50.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:49 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:49.867+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:50.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:49 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:49.929+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:50.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:49 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:49.929+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:50.540 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:50 smithi137 podman[69075]: 2024-04-25 15:13:50.275484055 +0000 UTC m=+5.450739522 container died e8149acb951f4628fe9c474d431d26557c9021f554fe1fcf0b3e2c7df8c7f3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T15:13:50.540 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:50 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:50.381+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:50.540 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:50 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:50.381+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:50.848 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:50 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:50.476+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:50.848 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:50 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:50.476+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:50.982 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:50 smithi137 podman[69075]: 2024-04-25 15:13:50.616862407 +0000 UTC m=+5.792117873 container cleanup e8149acb951f4628fe9c474d431d26557c9021f554fe1fcf0b3e2c7df8c7f3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T15:13:50.983 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:50 smithi137 bash[69075]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4 2024-04-25T15:13:50.983 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:50 smithi137 podman[69092]: 2024-04-25 15:13:50.784711 +0000 UTC m=+0.511894707 container remove e8149acb951f4628fe9c474d431d26557c9021f554fe1fcf0b3e2c7df8c7f3df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T15:13:51.118 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:50 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:50.846+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:51.118 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:50 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:50.846+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:51.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:50 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:50.832+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:51.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:50 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:50.832+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:51.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:50 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:50.894+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:51.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:50 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:50.894+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:51.436 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:51 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:51.116+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:51.463 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:51 smithi137 podman[69188]: 2024-04-25 15:13:51.129119438 +0000 UTC m=+0.018027734 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:13:51.464 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:51 smithi137 podman[69188]: 2024-04-25 15:13:51.25223623 +0000 UTC m=+0.141144525 container create 7948e811cd73cfc95966adcbb3145b8f6031bc666f0c6f529e937408b8d3dd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T15:13:51.464 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:51 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:51.202+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:51.464 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:51 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:51.360+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:51.465 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:51 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:51.360+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:51.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:51 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:51.434+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:51.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:51 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:51.434+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:51.732 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:51 smithi137 podman[69188]: 2024-04-25 15:13:51.463199314 +0000 UTC m=+0.352107610 container init 7948e811cd73cfc95966adcbb3145b8f6031bc666f0c6f529e937408b8d3dd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T15:13:51.732 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:51 smithi137 podman[69188]: 2024-04-25 15:13:51.466462652 +0000 UTC m=+0.355370950 container start 7948e811cd73cfc95966adcbb3145b8f6031bc666f0c6f529e937408b8d3dd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:13:51.733 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:51 smithi137 podman[69188]: 2024-04-25 15:13:51.563069571 +0000 UTC m=+0.451977877 container attach 7948e811cd73cfc95966adcbb3145b8f6031bc666f0c6f529e937408b8d3dd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T15:13:52.049 INFO:journalctl@ceph.osd.4.smithi137.stdout:Apr 25 15:13:51 smithi137 podman[69188]: 2024-04-25 15:13:51.757077198 +0000 UTC m=+0.645985498 container died 7948e811cd73cfc95966adcbb3145b8f6031bc666f0c6f529e937408b8d3dd3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T15:13:52.158 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:51 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:51.878+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:52.158 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:51 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:51.878+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:52.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:51.882+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:52.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:51.882+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:52.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:51.909+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:52.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:51 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:51.909+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:52.320 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:52 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:52.188+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:52.342 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T15:13:52.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:13:52.380 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T15:13:52.380 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T15:13:52.380 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.5 2024-04-25T15:13:52.408 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:52.156+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:52.675 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:52 smithi138 systemd[1]: Stopping Ceph osd.5 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:13:52.675 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:52.455+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:52.675 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:52.455+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:52.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:52 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:52.318+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:52.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:52 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:52.318+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:52.957 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:52.673+0000 7fa7a0570700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:13:52.957 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:52.673+0000 7fa7a0570700 -1 osd.5 100 *** Got signal Terminated *** 2024-04-25T15:13:52.957 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:52.673+0000 7fa7a0570700 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:13:52.957 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:52.863+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:52.958 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:52 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:52.863+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:53.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:52 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:52.870+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:53.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:52 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:52.870+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:53.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:52 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:52.889+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:53.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:52 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:52.889+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:53.453 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:53 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:53.144+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:53.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:53 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:53.162+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:53.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:53 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:53.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:53.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:53 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:53.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:53.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:53 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:53.451+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:53.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:53 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:53.451+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:54.155 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:53 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:53.866+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:54.156 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:53 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:53.866+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:54.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:53.856+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:54.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:53.856+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:54.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:53.908+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:54.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:53 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:53.908+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:54.431 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:54 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:54.154+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:54.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:54 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:54.129+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:54.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:54 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:54.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:54.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:54 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:54.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:54.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:54 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:54.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:54.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:54 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:54.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:54.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:54 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:54.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:13:55.131 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:54 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:54.820+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:55.131 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:54 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:54.820+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:55.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:54 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:54.898+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:55.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:54 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:54.898+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:55.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:54 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:54.842+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:55.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:54 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:54.842+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:55.417 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:55 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:55.130+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:55.482 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:55 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:55.324+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:55.482 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:55 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:55.324+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:55.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:55 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:55.109+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:55.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:55 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:55.109+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:13:55.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:55 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:55.416+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:55.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:55 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:55.416+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:55.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:55 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:55.416+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:13:56.116 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:55 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:55.796+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:56.117 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:55 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:55.796+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:56.117 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:55 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:55.796+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.335655+0000 front 2024-04-25T15:13:32.335675+0000 (oldest deadline 2024-04-25T15:13:55.235809+0000) 2024-04-25T15:13:56.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:55.849+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:56.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:55.849+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:56.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:55.871+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:56.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:55.871+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:56.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:55.871+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:13:56.392 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:56 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:56.115+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:56.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:56 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:56.301+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:56.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:56 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:56.301+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:56.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:56 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:56.102+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:56.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:56 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:56.102+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:13:56.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:56 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:56.391+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:56.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:56 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:56.391+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:56.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:56 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:56.391+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:13:57.096 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:56 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:56.812+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:11.733730+0000 front 2024-04-25T15:13:11.733583+0000 (oldest deadline 2024-04-25T15:13:40.534225+0000) 2024-04-25T15:13:57.097 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:56 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:56.812+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:20.534801+0000 front 2024-04-25T15:13:20.534902+0000 (oldest deadline 2024-04-25T15:13:46.434756+0000) 2024-04-25T15:13:57.097 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:56 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5[44992]: 2024-04-25T15:13:56.812+0000 7fa79cb64700 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.335655+0000 front 2024-04-25T15:13:32.335675+0000 (oldest deadline 2024-04-25T15:13:55.235809+0000) 2024-04-25T15:13:57.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:56.858+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:57.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:56.858+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:57.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:56.858+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:13:57.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:56.868+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:57.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:56.868+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:57.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:56 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:56.868+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:13:57.342 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:57 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:57.056+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:57.342 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:57 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:57.056+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:13:57.430 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:57 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:57.095+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:57.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:57 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:57.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:57.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:57 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:57.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:57.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:57 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:57.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:13:57.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:57 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:57.340+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:57.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:57 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:57.340+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:58.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:57.847+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:58.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:57.847+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:58.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:57.847+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:13:58.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:57.847+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:58.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:57.847+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:58.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:57 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:57.847+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:13:58.294 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:58 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:58.108+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:58.294 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:58 smithi138 podman[69770]: 2024-04-25 15:13:58.025268259 +0000 UTC m=+5.454434976 container died ee78f9788bb2f77e11c6a4c244089384a46bfa85b87e0d6000550225585cca6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T15:13:58.326 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:58.039+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:58.326 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:58.039+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:13:58.642 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:58 smithi138 podman[69770]: 2024-04-25 15:13:58.381740569 +0000 UTC m=+5.810907297 container cleanup ee78f9788bb2f77e11c6a4c244089384a46bfa85b87e0d6000550225585cca6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T15:13:58.642 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:58 smithi138 bash[69770]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5 2024-04-25T15:13:58.642 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:58 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:58.438+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:58.642 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:58 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:58.438+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:58.642 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:58 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:58.438+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:13:58.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:58.325+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:58.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:58.325+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:58.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:58 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:58.325+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:13:58.957 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:58 smithi138 podman[69787]: 2024-04-25 15:13:58.641255881 +0000 UTC m=+0.611481377 container remove ee78f9788bb2f77e11c6a4c244089384a46bfa85b87e0d6000550225585cca6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:13:59.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:58.851+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:13:59.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:58.851+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:13:59.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:58.851+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:13:59.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:58.855+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:13:59.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:58.855+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:13:59.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:58 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:58.855+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:13:59.311 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:13:59 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:13:59.062+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:13:59.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:58 smithi138 podman[69876]: 2024-04-25 15:13:58.901466996 +0000 UTC m=+0.018342790 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:13:59.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:59 smithi138 podman[69876]: 2024-04-25 15:13:59.017158154 +0000 UTC m=+0.134033947 container create 0797f9e7822114af15492cc578209ff3db77c1e67016c6582e2e03cb09cfedde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T15:13:59.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:59 smithi138 podman[69876]: 2024-04-25 15:13:59.236433097 +0000 UTC m=+0.353308891 container init 0797f9e7822114af15492cc578209ff3db77c1e67016c6582e2e03cb09cfedde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T15:13:59.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:59 smithi138 podman[69876]: 2024-04-25 15:13:59.239493929 +0000 UTC m=+0.356369727 container start 0797f9e7822114af15492cc578209ff3db77c1e67016c6582e2e03cb09cfedde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:13:59.312 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:59 smithi138 podman[69876]: 2024-04-25 15:13:59.311126958 +0000 UTC m=+0.428002766 container attach 0797f9e7822114af15492cc578209ff3db77c1e67016c6582e2e03cb09cfedde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:13:59.341 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:59 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:59.074+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:13:59.342 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:13:59 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:13:59.074+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:13:59.707 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:13:59 smithi138 podman[69876]: 2024-04-25 15:13:59.53214672 +0000 UTC m=+0.649022525 container died 0797f9e7822114af15492cc578209ff3db77c1e67016c6582e2e03cb09cfedde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:13:59.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:59 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:59.475+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:13:59.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:59 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:59.475+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:13:59.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:13:59 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:13:59.475+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:13:59.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:59 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:59.340+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:13:59.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:59 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:59.340+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:13:59.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:13:59 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:13:59.340+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:00.069 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 15:14:00 smithi138 podman[69903]: 2024-04-25 15:14:00.056102024 +0000 UTC m=+0.518425517 container remove 0797f9e7822114af15492cc578209ff3db77c1e67016c6582e2e03cb09cfedde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:14:00.070 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:00.026+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:00.070 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:00.026+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:00.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T15:14:00.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:00.154 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T15:14:00.155 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T15:14:00.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.6 2024-04-25T15:14:00.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:59.889+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:00.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:59.889+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:00.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:13:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:13:59.889+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:00.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:59.832+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:14:00.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:59.832+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:14:00.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:13:59 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:13:59.832+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:14:00.370 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:00.059+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:00.371 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:00.059+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:00.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:00.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:00.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:00.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:00.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:00 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:00.429+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:00.712 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:00 smithi094 systemd[1]: Stopping Ceph osd.6 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:00.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:00.369+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:00.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:00.369+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:00.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:00 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:00.369+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:01.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:00.883+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:01.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:00.883+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:01.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:00.883+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:01.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:00.720+0000 7f1dcfe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:01.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:00.720+0000 7f1dcfe00700 -1 osd.6 100 *** Got signal Terminated *** 2024-04-25T15:14:01.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:00.720+0000 7f1dcfe00700 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:14:01.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:00.795+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:14:01.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:00.795+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:14:01.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:00 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:00.796+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:14:01.452 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:01 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:01.061+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:01.452 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:01 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:01.061+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:01.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:01 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:01.094+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:01.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:01 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:01.094+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:01.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:01 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:01.341+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:01.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:01 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:01.341+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:01.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:01 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:01.341+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:01.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:01 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:01.450+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:01.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:01 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:01.450+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:01.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:01 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:01.450+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:02.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:01.818+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:14:02.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:01.818+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:14:02.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:01.818+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:14:02.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:01.920+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:02.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:01.920+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:02.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:01.920+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:02.447 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:02.086+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:02.447 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:02.086+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:02.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:02.136+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:02.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:02.136+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:02.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:02.292+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:02.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:02.292+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:02.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:02 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:02.292+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:02.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:02.445+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:02.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:02.445+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:02.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:02.445+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:02.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:02 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:02.445+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:03.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:02.877+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:03.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:02.877+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:03.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:02.877+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:03.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:02.860+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:14:03.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:02.860+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:14:03.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:02.860+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:14:03.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:02 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:02.860+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.180925+0000 front 2024-04-25T15:13:40.180886+0000 (oldest deadline 2024-04-25T15:14:01.880685+0000) 2024-04-25T15:14:03.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:03.115+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:03.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:03.115+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:03.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:03.115+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:03.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:03.170+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:03.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:03.170+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:03.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:03.306+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:03.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:03.306+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:03.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:03.306+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:03.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:03 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:03.306+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:03.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:03.473+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:03.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:03.473+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:03.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:03.473+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:03.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:03 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:03.473+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:04.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:03.907+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:04.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:03.907+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:04.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:03.907+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:04.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:03.830+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:14:04.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:03.830+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:14:04.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:03.830+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:14:04.214 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:03 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:03.830+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.180925+0000 front 2024-04-25T15:13:40.180886+0000 (oldest deadline 2024-04-25T15:14:01.880685+0000) 2024-04-25T15:14:04.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:04.133+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:04.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:04.133+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:04.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:04.133+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:04.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:04.131+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:04.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:04.131+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:04.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:04.131+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:04.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:04.339+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:04.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:04.339+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:04.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:04.339+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:04.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:04 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:04.339+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:04.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:04.479+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:04.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:04.479+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:04.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:04.479+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:04.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:04 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:04.479+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:05.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:04.790+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:12.171585+0000 front 2024-04-25T15:13:12.172566+0000 (oldest deadline 2024-04-25T15:13:36.871641+0000) 2024-04-25T15:14:05.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:04.790+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.773938+0000 front 2024-04-25T15:13:22.773748+0000 (oldest deadline 2024-04-25T15:13:48.073840+0000) 2024-04-25T15:14:05.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:04.790+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.175718+0000 front 2024-04-25T15:13:32.175812+0000 (oldest deadline 2024-04-25T15:13:55.075972+0000) 2024-04-25T15:14:05.213 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6[56086]: 2024-04-25T15:14:04.790+0000 7f1dc6600700 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.180925+0000 front 2024-04-25T15:13:40.180886+0000 (oldest deadline 2024-04-25T15:14:01.880685+0000) 2024-04-25T15:14:05.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:04.858+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:05.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:04.858+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:05.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:04.858+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:05.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:04.858+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:05.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:05.111+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:05.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:05.111+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:05.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:05.111+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:05.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:05.116+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:05.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:05.116+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:05.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:05.116+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:05.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:05.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:05.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:05.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:05.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:05.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:05.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:05 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:05.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:05.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:05.466+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:05.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:05.466+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:05.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:05.466+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:05.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:05 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:05.467+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:06.158 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:05 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:05.884+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:06.158 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:05 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:05.884+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:06.158 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:05 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:05.884+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:06.158 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:05 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:05.884+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:06.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:06.066+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:06.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:06.066+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:06.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:06.066+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:06.461 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:06 smithi094 podman[86542]: 2024-04-25 15:14:06.15656038 +0000 UTC m=+5.575957971 container died 739d48971d5d6009536660bdaff32871cd5bb3fdc164c9fff3e4249ab9bdbd5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T15:14:06.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:06.163+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:06.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:06.163+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:06.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:06.163+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:06.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:06.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:06.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:06.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:06.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:06.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:06.484 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:06 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:06.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:06.712 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:06 smithi094 podman[86542]: 2024-04-25 15:14:06.580239292 +0000 UTC m=+5.999636917 container cleanup 739d48971d5d6009536660bdaff32871cd5bb3fdc164c9fff3e4249ab9bdbd5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T15:14:06.713 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:06 smithi094 bash[86542]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6 2024-04-25T15:14:06.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:06.467+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:06.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:06.467+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:06.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:06.467+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:06.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:06 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:06.467+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:07.212 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:06 smithi094 podman[86561]: 2024-04-25 15:14:06.898779034 +0000 UTC m=+0.727441724 container remove 739d48971d5d6009536660bdaff32871cd5bb3fdc164c9fff3e4249ab9bdbd5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T15:14:07.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:06.862+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:07.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:06.862+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:07.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:06.862+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:07.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:06 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:06.863+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:07.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:07 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:07.038+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:07.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:07 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:07.038+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:07.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:07 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:07.038+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:07.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:07.208+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:07.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:07.208+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:07.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:07.208+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:07.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:07.357+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:07.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:07.357+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:07.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:07.357+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:07.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:07 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:07.357+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:07.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:07 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:07.484+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:07.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:07 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:07.484+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:07.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:07 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:07.484+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:07.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:07 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:07.484+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:08.096 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:07 smithi094 podman[86652]: 2024-04-25 15:14:07.719566858 +0000 UTC m=+0.037118708 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:14:08.096 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:07 smithi094 podman[86652]: 2024-04-25 15:14:07.833105177 +0000 UTC m=+0.150657032 container create b7a7c3b804acb7fa2a64a206f9f71041c496167e4ca4e1d8d87f1291ee3b2ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T15:14:08.096 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:07.866+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:08.097 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:07.866+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:08.097 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:07.866+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:08.097 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:07 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:07.866+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:08.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:08 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:08.061+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:08.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:08 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:08.061+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:08.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:08 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:08.061+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:08.462 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:08 smithi094 podman[86652]: 2024-04-25 15:14:08.095073102 +0000 UTC m=+0.412624941 container init b7a7c3b804acb7fa2a64a206f9f71041c496167e4ca4e1d8d87f1291ee3b2ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:14:08.462 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:08 smithi094 podman[86652]: 2024-04-25 15:14:08.106164044 +0000 UTC m=+0.423715926 container start b7a7c3b804acb7fa2a64a206f9f71041c496167e4ca4e1d8d87f1291ee3b2ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:14:08.463 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:08 smithi094 podman[86652]: 2024-04-25 15:14:08.185853003 +0000 UTC m=+0.503404865 container attach b7a7c3b804acb7fa2a64a206f9f71041c496167e4ca4e1d8d87f1291ee3b2ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-25T15:14:08.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:08 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:08.213+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:08.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:08 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:08.213+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:08.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:08 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:08.213+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:08.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:08 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:08.372+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:08.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:08 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:08.372+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:08.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:08 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:08.372+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:08.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:08 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:08.372+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:08.896 INFO:journalctl@ceph.osd.6.smithi094.stdout:Apr 25 15:14:08 smithi094 podman[86652]: 2024-04-25 15:14:08.54855394 +0000 UTC m=+0.866105792 container died b7a7c3b804acb7fa2a64a206f9f71041c496167e4ca4e1d8d87f1291ee3b2ace (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T15:14:08.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:08 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:08.522+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:08.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:08 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:08.522+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:08.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:08 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:08.522+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:08.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:08 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:08.522+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:09.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:08 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:08.898+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:09.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:08 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:08.898+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:09.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:08 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:08.898+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:09.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:08 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:08.898+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:09.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T15:14:09.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:09.408 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T15:14:09.408 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T15:14:09.408 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.7 2024-04-25T15:14:09.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:09.025+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:09.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:09.025+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:09.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:09.025+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:09.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:09.217+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:09.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:09.217+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:09.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:09.217+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:09.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:09 smithi137 systemd[1]: Stopping Ceph osd.7 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:09.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:09.351+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:09.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:09.351+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:09.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:09.351+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:09.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:09.351+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:09.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:09.539+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:09.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:09.539+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:09.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:09.539+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:09.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:09.539+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:09.982 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:09.726+0000 7faea4ecc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:09.983 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:09.726+0000 7faea4ecc700 -1 osd.7 100 *** Got signal Terminated *** 2024-04-25T15:14:09.983 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:09.726+0000 7faea4ecc700 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:14:10.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:09.892+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:10.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:09.892+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:10.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:09.892+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:10.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:09 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:09.892+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:10.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:09.989+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:10.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:09.989+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:10.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:09 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:09.989+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:10.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:10 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:10.194+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:10.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:10 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:10.194+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:10.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:10 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:10.194+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:10.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:10 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:10.362+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:10.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:10 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:10.362+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:10.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:10 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:10.362+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:10.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:10 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:10.362+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:10.729 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:10 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:10.547+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:10.729 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:10 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:10.547+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:10.729 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:10 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:10.547+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:10.729 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:10 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:10.547+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:11.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:10 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:10.893+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:11.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:10 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:10.893+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:11.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:10 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:10.893+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:11.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:10 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:10.893+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:11.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:10 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:10.984+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:11.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:10 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:10.984+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:11.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:10 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:10.984+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:11.482 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:11.403+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:11.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:11.403+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:11.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:11.403+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:11.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:11.403+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:11.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:11.226+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:11.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:11.226+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:11.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:11.226+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:11.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:11.226+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:49.341163+0000 front 2024-04-25T15:13:49.341216+0000 (oldest deadline 2024-04-25T15:14:11.041309+0000) 2024-04-25T15:14:11.942 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:11 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:11.501+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:11.942 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:11 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:11.501+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:11.942 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:11 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:11.501+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:11.942 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:11 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:11.501+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:12.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:11 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:11.940+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:12.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:11 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:11.940+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:12.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:11 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:11.940+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:12.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:11.869+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:12.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:11.869+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:12.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:11.869+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:12.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:11 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:11.869+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:12.733 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:12.262+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:12.733 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:12.262+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:12.733 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:12.262+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:12.733 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:12.262+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:49.341163+0000 front 2024-04-25T15:13:49.341216+0000 (oldest deadline 2024-04-25T15:14:11.041309+0000) 2024-04-25T15:14:12.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:12.421+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:12.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:12.421+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:12.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:12.421+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:12.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:12 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:12.421+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:12.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:12.544+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:12.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:12.544+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:12.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:12.544+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:12.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:12.544+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:12.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:12.544+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:13.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:12 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:12.832+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:13.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:12 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:12.832+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:13.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:12 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:12.832+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:13.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:12 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:12.832+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:13.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:12.980+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:13.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:12.980+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:13.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:12.980+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:13.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:12 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:12.980+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:13.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:13.214+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:13.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:13.214+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:13.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:13.214+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:13.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:13.214+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:49.341163+0000 front 2024-04-25T15:13:49.341216+0000 (oldest deadline 2024-04-25T15:14:11.041309+0000) 2024-04-25T15:14:13.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:13.420+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:13.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:13.420+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:13.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:13.420+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:13.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:13 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:13.420+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:13.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:13.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:13.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:13.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:13.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:13.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:13.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:13.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:13.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:13.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:14.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:13 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:13.809+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:14.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:13 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:13.809+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:14.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:13 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:13.809+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:14.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:13 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:13.809+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:14.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:13 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:13.809+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:14.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:13.987+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:14.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:13.987+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:14.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:13.987+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:14.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:13 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:13.987+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:14.482 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:14.217+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.238547+0000 front 2024-04-25T15:13:25.238574+0000 (oldest deadline 2024-04-25T15:13:51.138698+0000) 2024-04-25T15:14:14.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:14.217+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.139113+0000 front 2024-04-25T15:13:31.139175+0000 (oldest deadline 2024-04-25T15:13:54.639226+0000) 2024-04-25T15:14:14.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:14.217+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.540642+0000 front 2024-04-25T15:13:37.540417+0000 (oldest deadline 2024-04-25T15:14:03.440162+0000) 2024-04-25T15:14:14.483 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7[50870]: 2024-04-25T15:14:14.218+0000 7faea14c0700 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:49.341163+0000 front 2024-04-25T15:13:49.341216+0000 (oldest deadline 2024-04-25T15:14:11.041309+0000) 2024-04-25T15:14:14.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:14.375+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:14.483 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:14.375+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:14.484 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:14.375+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:14.484 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:14.375+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:14.484 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:14 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:14.375+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:14.948 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:14.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:14.949 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:14.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:14.949 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:14.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:14.949 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:14.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:14.949 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:14.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:15.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:14.947+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:15.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:14.947+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:15.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:14.947+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:15.208 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:14 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:14.947+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:15.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:14 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:14.821+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:15.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:14 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:14.821+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:15.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:14 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:14.821+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:15.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:14 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:14.821+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:15.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:14 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:14.821+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:15.551 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:15 smithi137 podman[69294]: 2024-04-25 15:14:15.235751384 +0000 UTC m=+5.618632821 container died dfcf59ca46ed75fc1d554afc68f7cfd71cccf121e4886f6e4ce99e47d57124b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:14:15.552 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:15 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:15.402+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:15.552 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:15 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:15.402+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:15.552 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:15 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:15.402+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:15.552 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:15 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:15.402+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:15.553 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:15 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:15.402+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:15.927 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:15.617+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:15.928 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:15.617+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:15.928 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:15.617+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:15.928 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:15.617+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:15.928 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:15.617+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:15.982 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:15 smithi137 podman[69294]: 2024-04-25 15:14:15.551223838 +0000 UTC m=+5.934105247 container cleanup dfcf59ca46ed75fc1d554afc68f7cfd71cccf121e4886f6e4ce99e47d57124b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T15:14:15.983 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:15 smithi137 bash[69294]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7 2024-04-25T15:14:15.983 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:15 smithi137 podman[69322]: 2024-04-25 15:14:15.76916927 +0000 UTC m=+0.526067900 container remove dfcf59ca46ed75fc1d554afc68f7cfd71cccf121e4886f6e4ce99e47d57124b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T15:14:16.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:15.927+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:16.207 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:15.927+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:16.208 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:15.927+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:16.208 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:15 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:15.927+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:16.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:15 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:15.799+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:16.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:15 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:15.799+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:16.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:15 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:15.799+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:16.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:15 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:15.799+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:16.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:15 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:15.799+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:16.431 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:16 smithi137 podman[69415]: 2024-04-25 15:14:16.075129695 +0000 UTC m=+0.018016486 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:14:16.431 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:16 smithi137 podman[69415]: 2024-04-25 15:14:16.203218739 +0000 UTC m=+0.146105505 container create 18c921805e310df191eb501810abb7777083638b395946ae6e1dea25fc3ced2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T15:14:16.431 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:16 smithi137 podman[69415]: 2024-04-25 15:14:16.430434086 +0000 UTC m=+0.373320853 container init 18c921805e310df191eb501810abb7777083638b395946ae6e1dea25fc3ced2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T15:14:16.431 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:16 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:16.378+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:16.432 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:16 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:16.378+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:16.432 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:16 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:16.378+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:16.432 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:16 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:16.378+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:16.432 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:16 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:16.378+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:16.732 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:16 smithi137 podman[69415]: 2024-04-25 15:14:16.433551014 +0000 UTC m=+0.376437786 container start 18c921805e310df191eb501810abb7777083638b395946ae6e1dea25fc3ced2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T15:14:16.733 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:16 smithi137 podman[69415]: 2024-04-25 15:14:16.522191848 +0000 UTC m=+0.465078623 container attach 18c921805e310df191eb501810abb7777083638b395946ae6e1dea25fc3ced2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T15:14:16.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:16.648+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:16.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:16.648+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:16.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:16.648+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:16.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:16.648+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:16.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:16.648+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:17.016 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:16 smithi137 podman[69415]: 2024-04-25 15:14:16.733308998 +0000 UTC m=+0.676195778 container died 18c921805e310df191eb501810abb7777083638b395946ae6e1dea25fc3ced2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:14:17.115 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:16 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:16.752+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:17.115 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:16 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:16.752+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:17.115 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:16 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:16.752+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:17.115 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:16 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:16.752+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:17.115 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:16 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:16.752+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:17.252 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T15:14:17.270 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:17 smithi137 podman[69441]: 2024-04-25 15:14:17.225406137 +0000 UTC m=+0.487189847 container remove 18c921805e310df191eb501810abb7777083638b395946ae6e1dea25fc3ced2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T15:14:17.270 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:17 smithi137 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-25T15:14:17.270 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:17 smithi137 systemd[1]: Stopped Ceph osd.7 for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T15:14:17.270 INFO:journalctl@ceph.osd.7.smithi137.stdout:Apr 25 15:14:17 smithi137 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.7.service: Consumed 7.825s CPU time. 2024-04-25T15:14:17.292 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:17.292 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T15:14:17.292 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T15:14:17.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.8 2024-04-25T15:14:17.320 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:16.970+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:17.320 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:16.970+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:17.320 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:16.970+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:17.320 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:16 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:16.970+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:17.592 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 systemd[1]: Stopping Ceph osd.8 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:17.673 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:17.590+0000 7f2ec01ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:17.673 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:17.590+0000 7f2ec01ca700 -1 osd.8 100 *** Got signal Terminated *** 2024-04-25T15:14:17.673 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:17.590+0000 7f2ec01ca700 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:14:17.673 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:17.670+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:17.674 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:17.670+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:17.674 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:17.670+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:17.674 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:17.670+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:17.674 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:17.670+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:17.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:17 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:17.392+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:17.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:17 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:17.392+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:17.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:17 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:17.392+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:17.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:17 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:17.392+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:17.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:17 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:17.392+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:17.813 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:17 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:17.748+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:17.814 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:17 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:17.748+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:17.814 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:17 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:17.748+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:17.814 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:17 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:17.748+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:17.814 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:17 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:17.748+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:18.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:17.989+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:18.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:17.989+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:18.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:17.989+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:18.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:17 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:17.989+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:18.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:18 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:18.407+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:18.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:18 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:18.407+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:18.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:18 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:18.407+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:18.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:18 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:18.407+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:18.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:18 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:18.407+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:19.035 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:18 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:18.712+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:19.036 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:18 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:18.712+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:19.036 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:18 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:18.712+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:19.036 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:18 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:18.712+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:19.036 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:18 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:18.712+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:19.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:18.779+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:19.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:18.779+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:19.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:18.779+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:19.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:18.779+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:19.214 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:18 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:18.779+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:19.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:19.034+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:19.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:19.034+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:19.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:19.034+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:19.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:19.034+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:19.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:19.034+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:52.760832+0000 front 2024-04-25T15:13:52.760855+0000 (oldest deadline 2024-04-25T15:14:18.660982+0000) 2024-04-25T15:14:19.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:19.388+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:19.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:19.388+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:19.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:19.388+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:19.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:19.388+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:19.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:19.388+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:19.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:19.665+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:19.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:19.665+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:19.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:19.665+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:19.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:19.665+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:19.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:19 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:19.665+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:20.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:19 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:19.768+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:20.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:19 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:19.768+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:20.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:19 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:19.768+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:20.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:19 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:19.768+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:20.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:19 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:19.768+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:20.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:20.011+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:20.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:20.011+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:20.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:20.011+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:20.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:20.011+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:20.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:20.011+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:52.760832+0000 front 2024-04-25T15:13:52.760855+0000 (oldest deadline 2024-04-25T15:14:18.660982+0000) 2024-04-25T15:14:20.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:20.358+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:20.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:20.358+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:20.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:20.358+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:20.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:20.358+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:20.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:20 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:20.358+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:20.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:20.638+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:20.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:20.638+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:20.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:20.638+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:20.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:20.638+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:20.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:20 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:20.638+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:21.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:20 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:20.765+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:21.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:20 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:20.765+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:21.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:20 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:20.765+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:21.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:20 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:20.765+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:21.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:20 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:20.765+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:21.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.043+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:21.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.043+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:21.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.043+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:21.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.043+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:21.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.043+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:52.760832+0000 front 2024-04-25T15:13:52.760855+0000 (oldest deadline 2024-04-25T15:14:18.660982+0000) 2024-04-25T15:14:21.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:21 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:21.384+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:21.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:21 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:21.384+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:21.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:21 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:21.384+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:21.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:21 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:21.384+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:21.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:21 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:21.384+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:21.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:21.649+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:21.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:21.649+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:21.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:21.649+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:21.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:21.649+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:21.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:21.649+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:21.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:21.649+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:22.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:21.756+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:22.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:21.756+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:22.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:21.756+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:22.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:21.756+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:22.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:21 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:21.756+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:22.457 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.997+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:24.057496+0000 front 2024-04-25T15:13:24.057617+0000 (oldest deadline 2024-04-25T15:13:49.357507+0000) 2024-04-25T15:14:22.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.997+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.058430+0000 front 2024-04-25T15:13:31.058549+0000 (oldest deadline 2024-04-25T15:13:59.259102+0000) 2024-04-25T15:14:22.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.997+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:39.259857+0000 front 2024-04-25T15:13:39.259766+0000 (oldest deadline 2024-04-25T15:14:02.159573+0000) 2024-04-25T15:14:22.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.997+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:47.460751+0000 front 2024-04-25T15:13:47.460648+0000 (oldest deadline 2024-04-25T15:14:12.760532+0000) 2024-04-25T15:14:22.458 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:21 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8[51722]: 2024-04-25T15:14:21.997+0000 7f2ebc7be700 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:52.760832+0000 front 2024-04-25T15:13:52.760855+0000 (oldest deadline 2024-04-25T15:14:18.660982+0000) 2024-04-25T15:14:22.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:22 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:22.342+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:22.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:22 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:22.342+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:22.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:22 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:22.342+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:22.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:22 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:22.342+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:22.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:22 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:22.342+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:22.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:22 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:22.342+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:22.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:22 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:22.651+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:22.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:22 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:22.651+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:22.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:22 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:22.651+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:22.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:22 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:22.651+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:22.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:22 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:22.651+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:22.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:22 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:22.651+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:23.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:22.744+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:23.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:22.744+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:23.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:22.744+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:23.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:22.744+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:23.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:22.744+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:23.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:22 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:22.744+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:23.259 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:22 smithi138 podman[69993]: 2024-04-25 15:14:22.993602618 +0000 UTC m=+5.508543875 container died fc91bdf21567b7e7eac36d9094fcc1dbd5aec556afcc8f2780a40a38f3db546e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:14:23.615 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:23 smithi138 podman[69993]: 2024-04-25 15:14:23.357502023 +0000 UTC m=+5.872443280 container cleanup fc91bdf21567b7e7eac36d9094fcc1dbd5aec556afcc8f2780a40a38f3db546e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:14:23.616 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:23 smithi138 bash[69993]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8 2024-04-25T15:14:23.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:23 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:23.385+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:23.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:23 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:23.385+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:23.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:23 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:23.385+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:23.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:23 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:23.385+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:23.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:23 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:23.385+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:23.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:23 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:23.385+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:23.957 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:23 smithi138 podman[70010]: 2024-04-25 15:14:23.616805193 +0000 UTC m=+0.618904715 container remove fc91bdf21567b7e7eac36d9094fcc1dbd5aec556afcc8f2780a40a38f3db546e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:14:23.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:23.614+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:23.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:23.614+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:23.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:23.614+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:23.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:23.614+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:23.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:23.614+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:23.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:23 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:23.614+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:23.962 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:23.699+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:23.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:23.699+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:23.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:23.699+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:23.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:23.699+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:23.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:23.699+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:23.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:23 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:23.699+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:24.237 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:23 smithi138 podman[70101]: 2024-04-25 15:14:23.880162657 +0000 UTC m=+0.017585835 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:14:24.237 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:23 smithi138 podman[70101]: 2024-04-25 15:14:23.98440042 +0000 UTC m=+0.121823567 container create 4e03071d86fa9934513670b2f88581aa1a94ab385e591f176368f42d4ca6da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:14:24.237 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:24 smithi138 podman[70101]: 2024-04-25 15:14:24.236784403 +0000 UTC m=+0.374207547 container init 4e03071d86fa9934513670b2f88581aa1a94ab385e591f176368f42d4ca6da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:14:24.534 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:24 smithi138 podman[70101]: 2024-04-25 15:14:24.239973973 +0000 UTC m=+0.377397115 container start 4e03071d86fa9934513670b2f88581aa1a94ab385e591f176368f42d4ca6da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T15:14:24.534 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:24 smithi138 podman[70101]: 2024-04-25 15:14:24.319928251 +0000 UTC m=+0.457351409 container attach 4e03071d86fa9934513670b2f88581aa1a94ab385e591f176368f42d4ca6da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T15:14:24.535 INFO:journalctl@ceph.osd.8.smithi138.stdout:Apr 25 15:14:24 smithi138 podman[70101]: 2024-04-25 15:14:24.533671786 +0000 UTC m=+0.671094929 container died 4e03071d86fa9934513670b2f88581aa1a94ab385e591f176368f42d4ca6da0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:14:24.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:24.414+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:24.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:24.414+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:24.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:24.414+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:24.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:24.414+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:24.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:24.414+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:24.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:24 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:24.414+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:24.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:24.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:24.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:24.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:24.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:24.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:24.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:24.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:24.959 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:24.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:24.959 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:24 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:24.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:25.071 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T15:14:25.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:25.150 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T15:14:25.150 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T15:14:25.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.9 2024-04-25T15:14:25.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:24.738+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:25.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:24.738+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:25.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:24.738+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:25.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:24.738+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:25.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:24.738+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:25.213 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:24 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:24.738+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:25.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:25.571+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:25.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:25.571+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:25.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:25.571+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:25.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:25.571+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:25.708 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:25.571+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:25.709 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:25.571+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:25.710 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 systemd[1]: Stopping Ceph osd.9 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:25.711 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.708+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:25.714 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.708+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:25.714 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.708+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:25.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:25.382+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:25.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:25.382+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:25.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:25.382+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:25.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:25.382+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:25.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:25.382+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:25.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:25 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:25.382+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:25.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.708+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:25.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.708+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:25.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.708+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:25.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.773+0000 7f9574400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:25.964 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.773+0000 7f9574400700 -1 osd.9 100 *** Got signal Terminated *** 2024-04-25T15:14:25.964 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:25 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:25.773+0000 7f9574400700 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:14:26.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:26.424+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:26.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:26.424+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:26.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:26.424+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:26.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:26.424+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:26.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:26.424+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:26.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:26.424+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:26.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:26 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:26.424+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:26.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:26.555+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:26.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:26.555+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:26.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:26.555+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:26.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:26.555+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:26.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:26.555+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:26.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:26 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:26.555+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:26.962 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:26.687+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:26.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:26.687+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:26.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:26.687+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:26.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:26.687+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:26.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:26.687+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:26.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:26 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:26.687+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:27.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:27.377+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:27.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:27.377+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:27.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:27.377+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:27.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:27.377+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:27.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:27.377+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:27.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:27.377+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:27.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:27 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:27.377+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:27.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:27.576+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:27.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:27.576+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:27.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:27.576+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:27.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:27.576+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:27.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:27.576+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:27.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:27 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:27.576+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:27.962 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:27 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:27.678+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:27.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:27 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:27.678+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:27.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:27 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:27.678+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:27.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:27 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:27.678+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:27.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:27 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:27.678+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:27.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:27 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:27.678+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:28.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:28.328+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:28.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:28.328+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:28.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:28.328+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:28.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:28.328+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:28.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:28.328+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:28.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:28.328+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:28.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:28 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:28.328+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:28.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:28.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:28.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:28.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:28.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:28.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:28.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:28.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:28.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:28.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:28.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:28 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:28.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:28.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:28 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:28.676+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:28.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:28 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:28.676+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:28.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:28 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:28.676+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:28.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:28 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:28.676+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:28.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:28 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:28.676+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:28.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:28 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:28.676+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:29.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:29.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:29.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:29.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:29.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:29.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:29.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:29.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:29.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:29.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:29.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:29.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:29.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:29 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:29.354+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:29.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:29.599+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:29.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:29.599+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:29.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:29.599+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:29.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:29.599+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:29.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:29.599+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:29.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:29.599+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:29.962 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:29 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:29.633+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:29.962 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:29 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:29.633+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:29.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:29 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:29.633+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:29.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:29 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:29.633+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:29.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:29 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:29.633+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:29.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:29 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:29.633+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:30.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:30.319+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:30.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:30.319+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:30.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:30.319+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:30.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:30.319+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:30.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:30.319+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:30.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:30.319+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:30.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:30 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:30.319+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:30.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:30 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:30.613+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:30.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:30 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:30.613+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:30.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:30 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:30.613+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:30.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:30 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:30.613+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:30.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:30 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:30.613+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:30.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:30 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:30.613+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:30.962 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:30 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:30.647+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:16.152012+0000 front 2024-04-25T15:13:16.152458+0000 (oldest deadline 2024-04-25T15:13:41.451495+0000) 2024-04-25T15:14:30.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:30 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:30.647+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:22.554120+0000 front 2024-04-25T15:13:22.553841+0000 (oldest deadline 2024-04-25T15:13:47.253503+0000) 2024-04-25T15:14:30.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:30 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:30.647+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:31.955688+0000 front 2024-04-25T15:13:31.955747+0000 (oldest deadline 2024-04-25T15:13:56.655950+0000) 2024-04-25T15:14:30.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:30 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:30.647+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:40.158408+0000 front 2024-04-25T15:13:40.158454+0000 (oldest deadline 2024-04-25T15:14:04.858414+0000) 2024-04-25T15:14:30.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:30 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:30.647+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.260739+0000 front 2024-04-25T15:13:48.261047+0000 (oldest deadline 2024-04-25T15:14:12.960707+0000) 2024-04-25T15:14:30.963 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:30 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9[62656]: 2024-04-25T15:14:30.647+0000 7f956ac00700 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:56.462186+0000 front 2024-04-25T15:13:56.462216+0000 (oldest deadline 2024-04-25T15:14:22.362146+0000) 2024-04-25T15:14:31.408 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:31 smithi094 podman[86769]: 2024-04-25 15:14:31.11011579 +0000 UTC m=+5.496866963 container died c33294b126e67a93083ea9705fde4b5a8731bf71efeb3f54cf46f9d82b56f576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T15:14:31.712 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:31 smithi094 podman[86769]: 2024-04-25 15:14:31.526242049 +0000 UTC m=+5.912993252 container cleanup c33294b126e67a93083ea9705fde4b5a8731bf71efeb3f54cf46f9d82b56f576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T15:14:31.713 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:31 smithi094 bash[86769]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9 2024-04-25T15:14:31.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:31 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:31.310+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:31.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:31 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:31.310+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:31.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:31 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:31.310+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:31.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:31 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:31.310+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:31.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:31 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:31.310+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:31.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:31 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:31.310+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:31.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:31 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:31.310+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:31.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:31 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:31.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:31.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:31 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:31.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:31.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:31 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:31.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:31.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:31 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:31.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:31.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:31 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:31.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:31.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:31 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:31.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:32.212 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:31 smithi094 podman[86785]: 2024-04-25 15:14:31.841030458 +0000 UTC m=+0.713367276 container remove c33294b126e67a93083ea9705fde4b5a8731bf71efeb3f54cf46f9d82b56f576 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T15:14:32.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:32 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:32.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:32.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:32 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:32.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:32.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:32 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:32.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:32.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:32 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:32.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:32.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:32 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:32.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:32.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:32 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:32.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:32.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:32 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:32.305+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:32.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:32 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:32.594+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:32.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:32 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:32.594+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:32.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:32 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:32.594+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:32.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:32 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:32.594+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:32.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:32 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:32.594+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:32.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:32 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:32.594+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:33.076 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:32 smithi094 podman[86874]: 2024-04-25 15:14:32.66364425 +0000 UTC m=+0.035819025 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:14:33.076 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:32 smithi094 podman[86874]: 2024-04-25 15:14:32.787236861 +0000 UTC m=+0.159411671 container create 80237a1140c97a47c47d4377bcdeaa58659a2bd8360a923dec78ed114d027070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-25T15:14:33.076 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:33 smithi094 podman[86874]: 2024-04-25 15:14:33.074594539 +0000 UTC m=+0.446769356 container init 80237a1140c97a47c47d4377bcdeaa58659a2bd8360a923dec78ed114d027070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T15:14:33.462 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:33 smithi094 podman[86874]: 2024-04-25 15:14:33.086020443 +0000 UTC m=+0.458195223 container start 80237a1140c97a47c47d4377bcdeaa58659a2bd8360a923dec78ed114d027070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:14:33.463 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:33 smithi094 podman[86874]: 2024-04-25 15:14:33.165090982 +0000 UTC m=+0.537265819 container attach 80237a1140c97a47c47d4377bcdeaa58659a2bd8360a923dec78ed114d027070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-25T15:14:33.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:33 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:33.265+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:33.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:33 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:33.265+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:33.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:33 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:33.265+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:33.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:33 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:33.265+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:33.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:33 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:33.265+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:33.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:33 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:33.265+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:33.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:33 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:33.265+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:33.819 INFO:journalctl@ceph.osd.9.smithi094.stdout:Apr 25 15:14:33 smithi094 podman[86874]: 2024-04-25 15:14:33.543841731 +0000 UTC m=+0.916016548 container died 80237a1140c97a47c47d4377bcdeaa58659a2bd8360a923dec78ed114d027070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:14:33.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:33 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:33.589+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:33.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:33 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:33.589+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:33.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:33 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:33.589+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:33.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:33 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:33.589+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:33.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:33 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:33.589+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:33.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:33 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:33.589+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:34.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T15:14:34.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:34.251 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T15:14:34.252 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T15:14:34.252 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.10 2024-04-25T15:14:34.578 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.266+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:34.579 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.266+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:34.579 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.266+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:34.579 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.266+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:34.579 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.266+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:34.579 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.266+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:34.579 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.266+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:34.580 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 systemd[1]: Stopping Ceph osd.10 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:34.580 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.576+0000 7fd95a988700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:34.580 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.576+0000 7fd95a988700 -1 osd.10 100 *** Got signal Terminated *** 2024-04-25T15:14:34.580 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:34 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:34.576+0000 7fd95a988700 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:14:34.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:34.619+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:34.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:34.619+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:34.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:34.619+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:34.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:34.619+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:34.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:34.619+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:34.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:34.619+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:35.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:35 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:35.287+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:35.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:35 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:35.287+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:35.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:35 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:35.287+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:35.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:35 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:35.287+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:35.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:35 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:35.287+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:35.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:35 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:35.287+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:35.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:35 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:35.287+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:35.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:35 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:35.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:35.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:35 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:35.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:35.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:35 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:35.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:35.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:35 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:35.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:35.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:35 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:35.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:35.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:35 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:35.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:36.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:36 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:36.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:36.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:36 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:36.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:36.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:36 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:36.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:36.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:36 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:36.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:36.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:36 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:36.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:36.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:36 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:36.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:36.734 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:36 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:36.330+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:36.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:36 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:36.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:36.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:36 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:36.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:36.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:36 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:36.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:36.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:36 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:36.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:36.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:36 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:36.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:36.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:36 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:36.577+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:37.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:37 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:37.368+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:37.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:37 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:37.368+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:37.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:37 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:37.368+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:37.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:37 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:37.368+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:37.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:37 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:37.368+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:37.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:37 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:37.368+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:37.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:37 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:37.368+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:37.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:37 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:37.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:37.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:37 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:37.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:37.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:37 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:37.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:37.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:37 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:37.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:37.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:37 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:37.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:37.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:37 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:37.605+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:38.634 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:38 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:38.333+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:38.634 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:38 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:38.333+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:38.634 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:38 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:38.333+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:38.634 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:38 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:38.333+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:38.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:38 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:38.333+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:38.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:38 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:38.333+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:38.635 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:38 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:38.333+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:38.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:38.578+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:38.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:38.578+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:38.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:38.578+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:38.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:38.578+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:38.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:38.578+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:38.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:38.578+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:39.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:39 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:39.364+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:15.464438+0000 front 2024-04-25T15:13:15.464499+0000 (oldest deadline 2024-04-25T15:13:40.164039+0000) 2024-04-25T15:14:39.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:39 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:39.364+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:21.864936+0000 front 2024-04-25T15:13:21.864974+0000 (oldest deadline 2024-04-25T15:13:46.565089+0000) 2024-04-25T15:14:39.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:39 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:39.364+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:32.366613+0000 front 2024-04-25T15:13:32.366691+0000 (oldest deadline 2024-04-25T15:13:57.666527+0000) 2024-04-25T15:14:39.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:39 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:39.364+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.667268+0000 front 2024-04-25T15:13:37.667141+0000 (oldest deadline 2024-04-25T15:14:02.967014+0000) 2024-04-25T15:14:39.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:39 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:39.364+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.768337+0000 front 2024-04-25T15:13:48.768385+0000 (oldest deadline 2024-04-25T15:14:13.468490+0000) 2024-04-25T15:14:39.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:39 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:39.364+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:57.569585+0000 front 2024-04-25T15:13:57.569487+0000 (oldest deadline 2024-04-25T15:14:22.269510+0000) 2024-04-25T15:14:39.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:39 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10[57762]: 2024-04-25T15:14:39.364+0000 7fd956f7c700 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-04-25T15:14:02.270396+0000 front 2024-04-25T15:14:02.270376+0000 (oldest deadline 2024-04-25T15:14:26.369885+0000) 2024-04-25T15:14:39.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:39 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:39.569+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:39.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:39 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:39.569+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:39.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:39 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:39.569+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:39.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:39 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:39.569+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:39.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:39 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:39.569+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:39.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:39 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:39.569+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:40.445 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:40 smithi137 podman[69521]: 2024-04-25 15:14:40.122982746 +0000 UTC m=+5.674962541 container died 43a0f1489bfd70e1fe15b4142225f29ba3f52a32f756fdb0652cd4b64e1bf540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T15:14:40.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:40 smithi137 podman[69521]: 2024-04-25 15:14:40.527042257 +0000 UTC m=+6.079022059 container cleanup 43a0f1489bfd70e1fe15b4142225f29ba3f52a32f756fdb0652cd4b64e1bf540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T15:14:40.733 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:40 smithi137 bash[69521]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10 2024-04-25T15:14:40.859 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:40.600+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:40.859 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:40.600+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:40.859 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:40.600+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:40.859 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:40.600+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:40.859 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:40.600+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:40.859 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:40.600+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:40.860 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:40 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:40.600+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:41.062 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:40 smithi137 podman[69550]: 2024-04-25 15:14:40.744663423 +0000 UTC m=+0.620673958 container remove 43a0f1489bfd70e1fe15b4142225f29ba3f52a32f756fdb0652cd4b64e1bf540 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T15:14:41.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:41 smithi137 podman[69644]: 2024-04-25 15:14:40.965397802 +0000 UTC m=+0.018287674 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:14:41.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:41 smithi137 podman[69644]: 2024-04-25 15:14:41.070221778 +0000 UTC m=+0.123111629 container create bf09172a2ea0fea0e803497f1a0f97e967c976952dece69844f799f11e3080be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:14:41.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:41 smithi137 podman[69644]: 2024-04-25 15:14:41.306443469 +0000 UTC m=+0.359333322 container init bf09172a2ea0fea0e803497f1a0f97e967c976952dece69844f799f11e3080be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T15:14:41.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:41 smithi137 podman[69644]: 2024-04-25 15:14:41.309420023 +0000 UTC m=+0.362309884 container start bf09172a2ea0fea0e803497f1a0f97e967c976952dece69844f799f11e3080be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T15:14:41.390 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:41 smithi137 podman[69644]: 2024-04-25 15:14:41.389542334 +0000 UTC m=+0.442432206 container attach bf09172a2ea0fea0e803497f1a0f97e967c976952dece69844f799f11e3080be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T15:14:41.732 INFO:journalctl@ceph.osd.10.smithi137.stdout:Apr 25 15:14:41 smithi137 podman[69644]: 2024-04-25 15:14:41.600129299 +0000 UTC m=+0.653019153 container died bf09172a2ea0fea0e803497f1a0f97e967c976952dece69844f799f11e3080be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:14:41.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:41.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:41.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:41.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:41.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:41.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:41.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:41.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:41.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:41.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:41.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:41.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:41.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:41 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:41.618+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:42.163 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T15:14:42.202 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:42.203 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T15:14:42.203 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T15:14:42.203 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.11 2024-04-25T15:14:42.457 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 systemd[1]: Stopping Ceph osd.11 for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:42.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.550+0000 7f4512df0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:42.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.550+0000 7f4512df0700 -1 osd.11 100 *** Got signal Terminated *** 2024-04-25T15:14:42.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.550+0000 7f4512df0700 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:14:42.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:42.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:42.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:42.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:42.959 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:42.959 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:42.959 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:42 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:42.570+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:43.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:43.561+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:43.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:43.561+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:43.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:43.561+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:43.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:43.561+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:43.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:43.561+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:43.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:43.561+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:43.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:43 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:43.561+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:44.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:44.574+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:44.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:44.574+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:44.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:44.574+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:44.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:44.574+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:44.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:44.574+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:44.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:44.574+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:44.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:44 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:44.574+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:45.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:45.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:45.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:45.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:45.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:45.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:45.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:45.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:45 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:45.554+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6820 osd.8 since back 2024-04-25T15:14:20.923414+0000 front 2024-04-25T15:14:20.923444+0000 (oldest deadline 2024-04-25T15:14:45.023468+0000) 2024-04-25T15:14:46.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:46.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:46.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:46.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:46.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:46.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:46.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:46.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:46 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:46.552+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6820 osd.8 since back 2024-04-25T15:14:20.923414+0000 front 2024-04-25T15:14:20.923444+0000 (oldest deadline 2024-04-25T15:14:45.023468+0000) 2024-04-25T15:14:47.957 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.504+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-04-25T15:13:13.513165+0000 front 2024-04-25T15:13:13.513270+0000 (oldest deadline 2024-04-25T15:13:39.413004+0000) 2024-04-25T15:14:47.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.504+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6804 osd.1 since back 2024-04-25T15:13:25.214591+0000 front 2024-04-25T15:13:25.214519+0000 (oldest deadline 2024-04-25T15:13:48.114437+0000) 2024-04-25T15:14:47.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.505+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6804 osd.2 since back 2024-04-25T15:13:28.615419+0000 front 2024-04-25T15:13:28.615389+0000 (oldest deadline 2024-04-25T15:13:53.915469+0000) 2024-04-25T15:14:47.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.505+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-04-25T15:13:37.916916+0000 front 2024-04-25T15:13:37.917064+0000 (oldest deadline 2024-04-25T15:14:02.016763+0000) 2024-04-25T15:14:47.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.505+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6812 osd.4 since back 2024-04-25T15:13:48.418220+0000 front 2024-04-25T15:13:48.418261+0000 (oldest deadline 2024-04-25T15:14:12.518118+0000) 2024-04-25T15:14:47.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.505+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6812 osd.5 since back 2024-04-25T15:13:53.018981+0000 front 2024-04-25T15:13:53.019024+0000 (oldest deadline 2024-04-25T15:14:21.219613+0000) 2024-04-25T15:14:47.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.505+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.137:6820 osd.7 since back 2024-04-25T15:14:11.622163+0000 front 2024-04-25T15:14:11.622278+0000 (oldest deadline 2024-04-25T15:14:40.422669+0000) 2024-04-25T15:14:47.958 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11[58318]: 2024-04-25T15:14:47.505+0000 7f450f3e4700 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.138:6820 osd.8 since back 2024-04-25T15:14:20.923414+0000 front 2024-04-25T15:14:20.923444+0000 (oldest deadline 2024-04-25T15:14:45.023468+0000) 2024-04-25T15:14:48.304 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:47 smithi138 podman[70230]: 2024-04-25 15:14:47.962927465 +0000 UTC m=+5.575524345 container died c8025cfa318c8d40454fbdc9b24223a20a3a15ff9d96c522e62a605dce8a3ca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T15:14:48.305 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:48 smithi138 podman[70230]: 2024-04-25 15:14:48.304162711 +0000 UTC m=+5.916759592 container cleanup c8025cfa318c8d40454fbdc9b24223a20a3a15ff9d96c522e62a605dce8a3ca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T15:14:48.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:48 smithi138 bash[70230]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11 2024-04-25T15:14:48.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:48 smithi138 podman[70249]: 2024-04-25 15:14:48.501188124 +0000 UTC m=+0.536142356 container remove c8025cfa318c8d40454fbdc9b24223a20a3a15ff9d96c522e62a605dce8a3ca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T15:14:49.207 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:48 smithi138 podman[70341]: 2024-04-25 15:14:48.820509919 +0000 UTC m=+0.017863401 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T15:14:49.208 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:48 smithi138 podman[70341]: 2024-04-25 15:14:48.926851571 +0000 UTC m=+0.124205044 container create eaa3c0a323f9f027854d1cbea8e6484d026261aae308515a34b67ca1eb5e5f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True) 2024-04-25T15:14:49.208 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 podman[70341]: 2024-04-25 15:14:49.162961749 +0000 UTC m=+0.360315236 container init eaa3c0a323f9f027854d1cbea8e6484d026261aae308515a34b67ca1eb5e5f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T15:14:49.208 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 podman[70341]: 2024-04-25 15:14:49.166311658 +0000 UTC m=+0.363665129 container start eaa3c0a323f9f027854d1cbea8e6484d026261aae308515a34b67ca1eb5e5f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T15:14:49.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 podman[70341]: 2024-04-25 15:14:49.224931632 +0000 UTC m=+0.422285128 container attach eaa3c0a323f9f027854d1cbea8e6484d026261aae308515a34b67ca1eb5e5f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T15:14:49.707 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 podman[70341]: 2024-04-25 15:14:49.453923112 +0000 UTC m=+0.651276582 container died eaa3c0a323f9f027854d1cbea8e6484d026261aae308515a34b67ca1eb5e5f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T15:14:49.985 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T15:14:50.008 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 podman[70367]: 2024-04-25 15:14:49.965486304 +0000 UTC m=+0.505541167 container remove eaa3c0a323f9f027854d1cbea8e6484d026261aae308515a34b67ca1eb5e5f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T15:14:50.008 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.11.service: Deactivated successfully. 2024-04-25T15:14:50.008 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 systemd[1]: Stopped Ceph osd.11 for f04d929c-0312-11ef-bc93-c7b262605968. 2024-04-25T15:14:50.009 INFO:journalctl@ceph.osd.11.smithi138.stdout:Apr 25 15:14:49 smithi138 systemd[1]: ceph-f04d929c-0312-11ef-bc93-c7b262605968@osd.11.service: Consumed 6.711s CPU time. 2024-04-25T15:14:50.023 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:50.024 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T15:14:50.024 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T15:14:50.024 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.a 2024-04-25T15:14:50.462 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 15:14:50 smithi094 systemd[1]: Stopping Ceph mds.a for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:50.962 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 15:14:50 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a[69525]: 2024-04-25T15:14:50.593+0000 7f1e03e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:50.963 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 15:14:50 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a[69525]: 2024-04-25T15:14:50.593+0000 7f1e03e00700 -1 mds.a *** got signal Terminated *** 2024-04-25T15:14:53.644 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 15:14:53 smithi094 podman[86990]: 2024-04-25 15:14:53.359639697 +0000 UTC m=+2.890262032 container died 8c712da2151967d7ce5986e3403b63c5fadc6b5d47842ee0992fd574334bd437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T15:14:53.962 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 15:14:53 smithi094 podman[86990]: 2024-04-25 15:14:53.746369176 +0000 UTC m=+3.276991513 container cleanup 8c712da2151967d7ce5986e3403b63c5fadc6b5d47842ee0992fd574334bd437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:14:53.963 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 15:14:53 smithi094 bash[86990]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a 2024-04-25T15:14:54.462 INFO:journalctl@ceph.mds.a.smithi094.stdout:Apr 25 15:14:53 smithi094 podman[87016]: 2024-04-25 15:14:53.979677917 +0000 UTC m=+0.607884795 container remove 8c712da2151967d7ce5986e3403b63c5fadc6b5d47842ee0992fd574334bd437 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T15:14:54.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T15:14:54.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:14:54.621 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T15:14:54.621 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T15:14:54.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.d 2024-04-25T15:14:55.212 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 15:14:54 smithi094 systemd[1]: Stopping Ceph mds.d for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:14:55.712 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 15:14:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d[68547]: 2024-04-25T15:14:55.256+0000 7ff62b600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:14:55.713 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 15:14:55 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d[68547]: 2024-04-25T15:14:55.256+0000 7ff62b600700 -1 mds.d *** got signal Terminated *** 2024-04-25T15:14:59.557 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 15:14:59 smithi094 podman[87122]: 2024-04-25 15:14:59.278047155 +0000 UTC m=+4.151322938 container died c99bf4f29c5a6dcf8dda0474442a8525f51d1294d95eaa0263dcaf7d6e090509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T15:14:59.888 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 15:14:59 smithi094 podman[87122]: 2024-04-25 15:14:59.675435377 +0000 UTC m=+4.548711141 container cleanup c99bf4f29c5a6dcf8dda0474442a8525f51d1294d95eaa0263dcaf7d6e090509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T15:14:59.889 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 15:14:59 smithi094 bash[87122]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d 2024-04-25T15:14:59.889 INFO:journalctl@ceph.mds.d.smithi094.stdout:Apr 25 15:14:59 smithi094 podman[87136]: 2024-04-25 15:14:59.885192863 +0000 UTC m=+0.596143908 container remove c99bf4f29c5a6dcf8dda0474442a8525f51d1294d95eaa0263dcaf7d6e090509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T15:15:00.382 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T15:15:00.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:00.485 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T15:15:00.485 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T15:15:00.485 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.g 2024-04-25T15:15:00.712 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:15:00 smithi094 systemd[1]: Stopping Ceph mds.g for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:01.462 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:15:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g[69135]: 2024-04-25T15:15:01.001+0000 7f69fe000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:01.463 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:15:01 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g[69135]: 2024-04-25T15:15:01.001+0000 7f69fe000700 -1 mds.g *** got signal Terminated *** 2024-04-25T15:15:02.944 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:15:02 smithi094 podman[87242]: 2024-04-25 15:15:02.587453778 +0000 UTC m=+1.701926433 container died 48077639c94e2e02c1a572b4f7a33fc1b80c418149f094115e99b5fd424638d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-25T15:15:03.212 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:15:03 smithi094 podman[87242]: 2024-04-25 15:15:03.063419839 +0000 UTC m=+2.177892548 container cleanup 48077639c94e2e02c1a572b4f7a33fc1b80c418149f094115e99b5fd424638d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T15:15:03.213 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:15:03 smithi094 bash[87242]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g 2024-04-25T15:15:03.681 INFO:journalctl@ceph.mds.g.smithi094.stdout:Apr 25 15:15:03 smithi094 podman[87257]: 2024-04-25 15:15:03.22274131 +0000 UTC m=+0.618258632 container remove 48077639c94e2e02c1a572b4f7a33fc1b80c418149f094115e99b5fd424638d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T15:15:03.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T15:15:03.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:03.832 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T15:15:03.832 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T15:15:03.832 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.j 2024-04-25T15:15:04.212 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 15:15:03 smithi094 systemd[1]: Stopping Ceph mds.j for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:04.712 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 15:15:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j[68201]: 2024-04-25T15:15:04.391+0000 7ff2b6200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:04.713 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 15:15:04 smithi094 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j[68201]: 2024-04-25T15:15:04.391+0000 7ff2b6200700 -1 mds.j *** got signal Terminated *** 2024-04-25T15:15:08.182 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 15:15:07 smithi094 podman[87362]: 2024-04-25 15:15:07.886742796 +0000 UTC m=+3.641800207 container died 7579c7a12259c7a1ec95f869a3867c97be748644aa29d84bea0b94da47fcb88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-25T15:15:08.462 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 15:15:08 smithi094 podman[87362]: 2024-04-25 15:15:08.293090471 +0000 UTC m=+4.048147932 container cleanup 7579c7a12259c7a1ec95f869a3867c97be748644aa29d84bea0b94da47fcb88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T15:15:08.462 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 15:15:08 smithi094 bash[87362]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j 2024-04-25T15:15:08.851 INFO:journalctl@ceph.mds.j.smithi094.stdout:Apr 25 15:15:08 smithi094 podman[87388]: 2024-04-25 15:15:08.594109072 +0000 UTC m=+0.687509308 container remove 7579c7a12259c7a1ec95f869a3867c97be748644aa29d84bea0b94da47fcb88f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T15:15:08.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T15:15:08.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:08.984 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T15:15:08.984 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T15:15:08.984 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.b 2024-04-25T15:15:09.268 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:15:09 smithi137 systemd[1]: Stopping Ceph mds.b for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:09.732 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:15:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b[63292]: 2024-04-25T15:15:09.267+0000 7f72ddfcb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:09.733 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:15:09 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b[63292]: 2024-04-25T15:15:09.267+0000 7f72ddfcb700 -1 mds.b *** got signal Terminated *** 2024-04-25T15:15:10.732 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:15:10 smithi137 podman[69750]: 2024-04-25 15:15:10.493467215 +0000 UTC m=+1.320046304 container died 0c3ed092aa3c85e66e143cf05f30c0e23500f04b8bf87524f4c8d32de774e357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:15:11.077 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:15:10 smithi137 podman[69750]: 2024-04-25 15:15:10.934035818 +0000 UTC m=+1.760614886 container cleanup 0c3ed092aa3c85e66e143cf05f30c0e23500f04b8bf87524f4c8d32de774e357 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T15:15:11.077 INFO:journalctl@ceph.mds.b.smithi137.stdout:Apr 25 15:15:10 smithi137 bash[69750]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-b 2024-04-25T15:15:11.283 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T15:15:11.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:11.319 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T15:15:11.319 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T15:15:11.319 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.e 2024-04-25T15:15:11.647 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 15:15:11 smithi137 systemd[1]: Stopping Ceph mds.e for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:11.982 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 15:15:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e[62662]: 2024-04-25T15:15:11.645+0000 7fb297315700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:11.982 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 15:15:11 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e[62662]: 2024-04-25T15:15:11.646+0000 7fb297315700 -1 mds.e *** got signal Terminated *** 2024-04-25T15:15:14.829 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 15:15:14 smithi137 podman[69884]: 2024-04-25 15:15:14.504884015 +0000 UTC m=+2.972925322 container died 513567bf6136bb7e26a12f86a986179c0204f8730e950403f7576c8bb1646bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T15:15:14.830 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 15:15:14 smithi137 podman[69884]: 2024-04-25 15:15:14.828814804 +0000 UTC m=+3.296856106 container cleanup 513567bf6136bb7e26a12f86a986179c0204f8730e950403f7576c8bb1646bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:15:15.227 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 15:15:14 smithi137 bash[69884]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e 2024-04-25T15:15:15.228 INFO:journalctl@ceph.mds.e.smithi137.stdout:Apr 25 15:15:15 smithi137 podman[69898]: 2024-04-25 15:15:15.00490338 +0000 UTC m=+0.496353697 container remove 513567bf6136bb7e26a12f86a986179c0204f8730e950403f7576c8bb1646bde (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T15:15:15.242 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T15:15:15.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:15.275 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T15:15:15.275 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T15:15:15.275 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.h 2024-04-25T15:15:15.583 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 15:15:15 smithi137 systemd[1]: Stopping Ceph mds.h for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:15.982 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 15:15:15 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h[62969]: 2024-04-25T15:15:15.582+0000 7f3cf1e09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:15.982 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 15:15:15 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h[62969]: 2024-04-25T15:15:15.582+0000 7f3cf1e09700 -1 mds.h *** got signal Terminated *** 2024-04-25T15:15:19.124 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 15:15:18 smithi137 podman[70006]: 2024-04-25 15:15:18.801309049 +0000 UTC m=+3.316607212 container died 4976fcf0fff1e83c0abf055796c208fbefeacbe2ece0410ba159774b5182e51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T15:15:19.125 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 15:15:19 smithi137 podman[70006]: 2024-04-25 15:15:19.124243202 +0000 UTC m=+3.639541373 container cleanup 4976fcf0fff1e83c0abf055796c208fbefeacbe2ece0410ba159774b5182e51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:15:19.482 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 15:15:19 smithi137 bash[70006]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h 2024-04-25T15:15:19.483 INFO:journalctl@ceph.mds.h.smithi137.stdout:Apr 25 15:15:19 smithi137 podman[70020]: 2024-04-25 15:15:19.30871932 +0000 UTC m=+0.501654252 container remove 4976fcf0fff1e83c0abf055796c208fbefeacbe2ece0410ba159774b5182e51f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T15:15:19.551 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T15:15:19.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:19.585 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T15:15:19.586 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T15:15:19.586 DEBUG:teuthology.orchestra.run.smithi137:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.k 2024-04-25T15:15:19.894 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 15:15:19 smithi137 systemd[1]: Stopping Ceph mds.k for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:20.232 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 15:15:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k[62342]: 2024-04-25T15:15:19.893+0000 7f35b7103700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:20.233 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 15:15:19 smithi137 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k[62342]: 2024-04-25T15:15:19.893+0000 7f35b7103700 -1 mds.k *** got signal Terminated *** 2024-04-25T15:15:24.233 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 15:15:23 smithi137 podman[70126]: 2024-04-25 15:15:23.899565624 +0000 UTC m=+4.102499728 container died a93f09d911cc5bfd68fba86051a513547bd85bae4ac2220656c356fd5cf16441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True) 2024-04-25T15:15:24.628 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 15:15:24 smithi137 podman[70126]: 2024-04-25 15:15:24.385719718 +0000 UTC m=+4.588653808 container cleanup a93f09d911cc5bfd68fba86051a513547bd85bae4ac2220656c356fd5cf16441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:15:24.629 INFO:journalctl@ceph.mds.k.smithi137.stdout:Apr 25 15:15:24 smithi137 bash[70126]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-k 2024-04-25T15:15:24.755 DEBUG:teuthology.orchestra.run.smithi137:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T15:15:24.785 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:24.786 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T15:15:24.786 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T15:15:24.786 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.c 2024-04-25T15:15:25.082 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 15:15:24 smithi138 systemd[1]: Stopping Ceph mds.c for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:25.457 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 15:15:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c[63708]: 2024-04-25T15:15:25.080+0000 7fe5dc19b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:25.457 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 15:15:25 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c[63708]: 2024-04-25T15:15:25.080+0000 7fe5dc19b700 -1 mds.c *** got signal Terminated *** 2024-04-25T15:15:28.307 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 15:15:28 smithi138 podman[70450]: 2024-04-25 15:15:28.003641595 +0000 UTC m=+3.027216992 container died b76cdd18b0ed3e6e7912ecd129945d0301eab02e87c693fd37a734eb5c094545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T15:15:28.616 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 15:15:28 smithi138 podman[70450]: 2024-04-25 15:15:28.389156632 +0000 UTC m=+3.412732022 container cleanup b76cdd18b0ed3e6e7912ecd129945d0301eab02e87c693fd37a734eb5c094545 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T15:15:28.616 INFO:journalctl@ceph.mds.c.smithi138.stdout:Apr 25 15:15:28 smithi138 bash[70450]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-c 2024-04-25T15:15:28.738 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T15:15:28.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:28.773 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T15:15:28.774 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T15:15:28.774 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.f 2024-04-25T15:15:28.957 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 15:15:28 smithi138 systemd[1]: Stopping Ceph mds.f for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:29.457 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 15:15:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f[63400]: 2024-04-25T15:15:29.084+0000 7f50317e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:29.457 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 15:15:29 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f[63400]: 2024-04-25T15:15:29.084+0000 7f50317e8700 -1 mds.f *** got signal Terminated *** 2024-04-25T15:15:33.427 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 15:15:33 smithi138 podman[70582]: 2024-04-25 15:15:33.100519042 +0000 UTC m=+4.116483549 container died c20d22a287a8585f5c50f35213975203823068fd8ae37b7556ff4161155dcdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T15:15:33.707 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 15:15:33 smithi138 podman[70582]: 2024-04-25 15:15:33.525648337 +0000 UTC m=+4.541612849 container cleanup c20d22a287a8585f5c50f35213975203823068fd8ae37b7556ff4161155dcdcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T15:15:33.707 INFO:journalctl@ceph.mds.f.smithi138.stdout:Apr 25 15:15:33 smithi138 bash[70582]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-f 2024-04-25T15:15:33.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T15:15:33.987 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:33.988 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T15:15:33.988 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T15:15:33.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.i 2024-04-25T15:15:34.207 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 15:15:34 smithi138 systemd[1]: Stopping Ceph mds.i for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:34.707 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 15:15:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i[64019]: 2024-04-25T15:15:34.295+0000 7fd07c02b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:34.707 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 15:15:34 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i[64019]: 2024-04-25T15:15:34.295+0000 7fd07c02b700 -1 mds.i *** got signal Terminated *** 2024-04-25T15:15:37.940 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 15:15:37 smithi138 podman[70702]: 2024-04-25 15:15:37.665180235 +0000 UTC m=+3.466285086 container died 84824fb92537d9fe8997dbba64a46327d5f3a4c657974814ad3a5740b979c12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:15:38.207 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 15:15:38 smithi138 podman[70702]: 2024-04-25 15:15:38.055350128 +0000 UTC m=+3.856454979 container cleanup 84824fb92537d9fe8997dbba64a46327d5f3a4c657974814ad3a5740b979c12b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-25T15:15:38.295 INFO:journalctl@ceph.mds.i.smithi138.stdout:Apr 25 15:15:38 smithi138 bash[70702]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-i 2024-04-25T15:15:38.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T15:15:38.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:38.503 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T15:15:38.504 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T15:15:38.504 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.l 2024-04-25T15:15:38.957 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 15:15:38 smithi138 systemd[1]: Stopping Ceph mds.l for f04d929c-0312-11ef-bc93-c7b262605968... 2024-04-25T15:15:38.957 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 15:15:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l[63093]: 2024-04-25T15:15:38.809+0000 7f8884a17700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:15:38.957 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 15:15:38 smithi138 ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l[63093]: 2024-04-25T15:15:38.809+0000 7f8884a17700 -1 mds.l *** got signal Terminated *** 2024-04-25T15:15:41.651 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 15:15:41 smithi138 podman[70823]: 2024-04-25 15:15:41.319054876 +0000 UTC m=+2.602280316 container died e7ad3ca004a585f30b11b1a2e6a74c7d462397785f4152169d546e569444d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T15:15:41.652 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 15:15:41 smithi138 podman[70823]: 2024-04-25 15:15:41.650964046 +0000 UTC m=+2.934189462 container cleanup e7ad3ca004a585f30b11b1a2e6a74c7d462397785f4152169d546e569444d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T15:15:41.957 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 15:15:41 smithi138 bash[70823]: ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l 2024-04-25T15:15:41.957 INFO:journalctl@ceph.mds.l.smithi138.stdout:Apr 25 15:15:41 smithi138 podman[70837]: 2024-04-25 15:15:41.860193152 +0000 UTC m=+0.538634487 container remove e7ad3ca004a585f30b11b1a2e6a74c7d462397785f4152169d546e569444d54d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-f04d929c-0312-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T15:15:42.078 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f04d929c-0312-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T15:15:42.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T15:15:42.109 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T15:15:42.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f04d929c-0312-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T15:15:42.424 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T15:15:47.443 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f04d929c-0312-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T15:15:47.643 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T15:15:49.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f04d929c-0312-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T15:15:50.183 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T15:15:52.495 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:15:52.586 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:15:52.617 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:15:52.644 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T15:15:52.646 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi094/crash 2024-04-25T15:15:52.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash -- . 2024-04-25T15:15:52.741 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T15:15:52.742 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-04-25T15:15:52.746 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi137/crash 2024-04-25T15:15:52.747 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash -- . 2024-04-25T15:15:52.803 INFO:teuthology.orchestra.run.smithi137.stderr:tar: /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T15:15:52.803 INFO:teuthology.orchestra.run.smithi137.stderr:tar: Error is not recoverable: exiting now 2024-04-25T15:15:52.804 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi138/crash 2024-04-25T15:15:52.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash -- . 2024-04-25T15:15:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/f04d929c-0312-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T15:15:52.836 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-04-25T15:15:52.837 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T15:15:52.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T15:15:52.939 INFO:tasks.cephadm:Compressing logs... 2024-04-25T15:15:52.940 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T15:15:52.944 DEBUG:teuthology.orchestra.run.smithi137:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T15:15:52.946 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T15:15:52.971 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log 2024-04-25T15:15:52.971 INFO:teuthology.orchestra.run.smithi138.stderr:: No such file or directory 2024-04-25T15:15:52.971 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T15:15:52.972 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T15:15:52.972 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T15:15:52.972 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T15:15:52.972 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T15:15:52.972 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T15:15:52.983 INFO:teuthology.orchestra.run.smithi137.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log 2024-04-25T15:15:52.983 INFO:teuthology.orchestra.run.smithi137.stderr:: No such file or directory 2024-04-25T15:15:52.983 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T15:15:52.983 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T15:15:52.983 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T15:15:52.984 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T15:15:52.984 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T15:15:52.984 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T15:15:52.984 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T15:15:52.984 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- 88.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T15:15:52.984 INFO:teuthology.orchestra.run.smithi137.stderr: /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T15:15:52.984 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T15:15:52.985 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.10.log: /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T15:15:52.985 INFO:teuthology.orchestra.run.smithi137.stderr: 94.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T15:15:52.985 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T15:15:52.985 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T15:15:52.985 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log: /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.e.log: 83.8% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T15:15:52.985 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T15:15:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T15:15:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T15:15:52.986 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T15:15:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T15:15:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T15:15:52.986 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.5% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr: 83.8% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr: 94.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T15:15:52.987 INFO:teuthology.orchestra.run.smithi138.stderr: 92.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T15:15:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log: 91.9% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T15:15:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T15:15:52.989 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.b.log: 94.3% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T15:15:52.991 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose 91.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T15:15:52.991 INFO:teuthology.orchestra.run.smithi138.stderr: -- /var/log/ceph/cephadm.log 2024-04-25T15:15:52.993 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T15:15:52.996 INFO:teuthology.orchestra.run.smithi138.stderr: 94.6% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T15:15:52.997 INFO:teuthology.orchestra.run.smithi137.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T15:15:52.997 INFO:teuthology.orchestra.run.smithi137.stderr: 91.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T15:15:53.000 INFO:teuthology.orchestra.run.smithi137.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T15:15:53.001 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory 2024-04-25T15:15:53.001 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log 2024-04-25T15:15:53.001 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T15:15:53.003 INFO:teuthology.orchestra.run.smithi137.stderr: 94.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T15:15:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T15:15:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T15:15:53.005 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.g.log: 88.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T15:15:53.005 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T15:15:53.007 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T15:15:53.007 INFO:teuthology.orchestra.run.smithi094.stderr: 90.2% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T15:15:53.007 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T15:15:53.008 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T15:15:53.009 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T15:15:53.010 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T15:15:53.011 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T15:15:53.013 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T15:15:53.014 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T15:15:53.017 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T15:15:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T15:15:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T15:15:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T15:15:53.027 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.4%/var/log/ceph/cephadm.log: -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T15:15:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.4% 90.9% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T15:15:53.038 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T15:15:53.040 INFO:teuthology.orchestra.run.smithi094.stderr: 93.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T15:15:53.095 INFO:teuthology.orchestra.run.smithi137.stderr: 93.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T15:15:53.096 INFO:teuthology.orchestra.run.smithi094.stderr: 94.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T15:15:53.319 INFO:teuthology.orchestra.run.smithi138.stderr: 93.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T15:15:53.382 INFO:teuthology.orchestra.run.smithi137.stderr: 92.6% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T15:15:53.385 INFO:teuthology.orchestra.run.smithi137.stderr: 91.9% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T15:15:53.388 INFO:teuthology.orchestra.run.smithi138.stderr: 92.8% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T15:15:53.390 INFO:teuthology.orchestra.run.smithi094.stderr: 90.0% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T15:15:53.393 INFO:teuthology.orchestra.run.smithi138.stderr: 92.6% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T15:15:53.409 INFO:teuthology.orchestra.run.smithi137.stderr: 92.9% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T15:15:53.413 INFO:teuthology.orchestra.run.smithi138.stderr: 91.8% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T15:15:53.460 INFO:teuthology.orchestra.run.smithi094.stderr: 92.8% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T15:15:53.541 INFO:teuthology.orchestra.run.smithi094.stderr: 92.9% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T15:15:53.563 INFO:teuthology.orchestra.run.smithi094.stderr: 92.7% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T15:15:53.570 INFO:teuthology.orchestra.run.smithi137.stderr: 92.8% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T15:15:53.614 INFO:teuthology.orchestra.run.smithi094.stderr: 92.9% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T15:15:53.639 INFO:teuthology.orchestra.run.smithi138.stderr: 92.8% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T15:15:53.641 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-25T15:15:53.642 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m0.681s 2024-04-25T15:15:53.642 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m2.258s 2024-04-25T15:15:53.642 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.103s 2024-04-25T15:15:53.673 INFO:teuthology.orchestra.run.smithi137.stderr: 92.9% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T15:15:53.675 INFO:teuthology.orchestra.run.smithi137.stderr: 2024-04-25T15:15:53.675 INFO:teuthology.orchestra.run.smithi137.stderr:real 0m0.714s 2024-04-25T15:15:53.675 INFO:teuthology.orchestra.run.smithi137.stderr:user 0m2.629s 2024-04-25T15:15:53.675 INFO:teuthology.orchestra.run.smithi137.stderr:sys 0m0.111s 2024-04-25T15:15:53.962 INFO:teuthology.orchestra.run.smithi094.stderr: 90.4% -- replaced with /var/log/ceph/f04d929c-0312-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T15:15:53.965 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-04-25T15:15:53.965 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m0.989s 2024-04-25T15:15:53.965 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m3.324s 2024-04-25T15:15:53.966 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.347s 2024-04-25T15:15:53.967 INFO:tasks.cephadm:Archiving logs... 2024-04-25T15:15:53.972 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi094/log 2024-04-25T15:15:53.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T15:15:54.520 DEBUG:teuthology.misc:Transferring archived files from smithi137:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi137/log 2024-04-25T15:15:54.520 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T15:15:54.930 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi138/log 2024-04-25T15:15:54.931 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T15:15:55.296 INFO:tasks.cephadm:Removing cluster... 2024-04-25T15:15:55.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f04d929c-0312-11ef-bc93-c7b262605968 --force 2024-04-25T15:15:55.609 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T15:15:56.187 DEBUG:teuthology.orchestra.run.smithi137:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f04d929c-0312-11ef-bc93-c7b262605968 --force 2024-04-25T15:15:56.377 INFO:teuthology.orchestra.run.smithi137.stdout:Deleting cluster with fsid: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T15:15:56.626 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f04d929c-0312-11ef-bc93-c7b262605968 --force 2024-04-25T15:15:56.804 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: f04d929c-0312-11ef-bc93-c7b262605968 2024-04-25T15:15:57.061 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T15:15:57.061 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T15:15:57.119 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T15:15:57.135 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T15:15:57.150 INFO:tasks.cephadm:Teardown complete 2024-04-25T15:15:57.151 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T15:15:57.163 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T15:15:57.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T15:15:57.171 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T15:15:57.177 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T15:15:57.325 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T15:15:57.336 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T15:15:57.336 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:15:57.344 DEBUG:teuthology.orchestra.run.smithi137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:15:57.347 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:15:57.358 INFO:teuthology.orchestra.run.smithi137.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:15:57.360 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:15:57.364 INFO:teuthology.orchestra.run.smithi137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:15:57.364 INFO:teuthology.orchestra.run.smithi137.stdout:=============================================================================== 2024-04-25T15:15:57.364 INFO:teuthology.orchestra.run.smithi137.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 +937us[ +935us] +/- 45ms 2024-04-25T15:15:57.364 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -1347us[-1350us] +/- 66ms 2024-04-25T15:15:57.364 INFO:teuthology.orchestra.run.smithi137.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +15us[ +12us] +/- 74ms 2024-04-25T15:15:57.365 INFO:teuthology.orchestra.run.smithi137.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:15:57.366 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:15:57.366 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T15:15:57.366 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +910us[ +912us] +/- 45ms 2024-04-25T15:15:57.366 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -1367us[-1367us] +/- 66ms 2024-04-25T15:15:57.366 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -9400ns[-9400ns] +/- 74ms 2024-04-25T15:15:57.366 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:15:57.394 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:15:57.404 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:15:57.405 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-04-25T15:15:57.405 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 7 377 57 +959us[ +965us] +/- 45ms 2024-04-25T15:15:57.405 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -1364us[-1364us] +/- 66ms 2024-04-25T15:15:57.405 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 55 +39us[ +39us] +/- 74ms 2024-04-25T15:15:57.405 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:15:57.406 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T15:15:57.417 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T15:15:57.418 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T15:15:57.438 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T15:15:57.622 DEBUG:teuthology.orchestra.run.smithi137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T15:15:57.688 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T15:15:57.753 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:15:57.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:15:57.843 DEBUG:teuthology.orchestra.run.smithi137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:15:57.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:15:57.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:15:57.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:15:57.898 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T15:15:57.909 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T15:15:57.956 INFO:teuthology.task.internal:Duration was 2045.896807 seconds 2024-04-25T15:15:57.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T15:15:57.980 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T15:15:57.980 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T15:15:57.988 DEBUG:teuthology.orchestra.run.smithi137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T15:15:57.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T15:15:58.025 INFO:teuthology.orchestra.run.smithi137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:15:58.027 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:15:58.125 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:15:58.546 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T15:15:58.547 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-04-25T15:15:58.547 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T15:15:58.613 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi137.front.sepia.ceph.com 2024-04-25T15:15:58.614 DEBUG:teuthology.orchestra.run.smithi137:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T15:15:58.634 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T15:15:58.772 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T15:15:58.794 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T15:15:58.794 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T15:15:58.801 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T15:15:58.804 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T15:15:58.874 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T15:15:58.875 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T15:15:58.880 DEBUG:teuthology.orchestra.run.smithi137:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T15:15:58.882 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T15:15:59.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T15:15:59.403 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T15:15:59.404 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T15:15:59.493 DEBUG:teuthology.orchestra.run.smithi137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T15:15:59.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T15:15:59.542 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T15:15:59.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:15:59.562 DEBUG:teuthology.orchestra.run.smithi137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:15:59.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:15:59.585 INFO:teuthology.orchestra.run.smithi137.stdout:kernel.core_pattern = core 2024-04-25T15:15:59.605 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-25T15:15:59.653 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-04-25T15:15:59.697 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:15:59.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:15:59.744 DEBUG:teuthology.orchestra.run.smithi137:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:15:59.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:15:59.759 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T15:15:59.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:15:59.773 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T15:15:59.786 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T15:15:59.786 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi094 2024-04-25T15:15:59.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T15:15:59.892 DEBUG:teuthology.misc:Transferring archived files from smithi137:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi137 2024-04-25T15:15:59.892 DEBUG:teuthology.orchestra.run.smithi137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T15:15:59.927 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510/remote/smithi138 2024-04-25T15:15:59.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T15:15:59.963 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T15:15:59.963 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T15:15:59.971 DEBUG:teuthology.orchestra.run.smithi137:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T15:15:59.973 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T15:16:00.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T15:16:00.033 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T15:16:00.033 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T15:16:00.045 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T15:16:00.045 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T15:16:00.052 DEBUG:teuthology.orchestra.run.smithi137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T15:16:00.055 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T15:16:00.066 INFO:teuthology.orchestra.run.smithi137.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 15:15 /home/ubuntu/cephtest 2024-04-25T15:16:00.070 INFO:teuthology.orchestra.run.smithi138.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 15:16 /home/ubuntu/cephtest 2024-04-25T15:16:00.100 INFO:teuthology.orchestra.run.smithi094.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 15:16 /home/ubuntu/cephtest 2024-04-25T15:16:00.104 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T15:16:00.117 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T15:16:00.148 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T15:16:00.170 DEBUG:teuthology.nuke:shortname: smithi094 2024-04-25T15:16:00.171 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T15:16:00.192 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:32:38.968570', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG1z9CW5EAaB6BDadkeQ2MgtgSQXOHIq7VhMO0e6RHkkXxWjhAeygrR5NWKHONpGKG0guLfmkTAVkQYxUlswH78='} 2024-04-25T15:16:00.197 DEBUG:teuthology.nuke:shortname: smithi137 2024-04-25T15:16:00.198 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T15:16:00.223 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi137.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:32:38.970031', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCDgTMsZivjEeq1T7k4rVha/1I/6nTMwHxMKLtWGcGc2P9DlZId7M4vylD8taa9Tte76dWn35Ar84+rkMt6kP2U='} 2024-04-25T15:16:00.224 DEBUG:teuthology.nuke:shortname: smithi138 2024-04-25T15:16:00.225 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T15:16:00.253 INFO:teuthology.orchestra.console.smithi094:Power off 2024-04-25T15:16:00.254 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T15:16:00.267 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_13:50:46-fs-wip-lusov-quiescer-distro-default-smithi/7673510', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 14:32:38.971293', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEBSpKTrMbiaLLsuseKIzrGbGKf2e9kdMnDaAyKsyE8Iw06v1WU2yAqDycvyR6ZuCnt7JLXURvdZmjdUfcbRubQ='} 2024-04-25T15:16:00.284 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-04-25T15:16:00.285 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:00.309 INFO:teuthology.orchestra.console.smithi137:Power off 2024-04-25T15:16:00.310 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T15:16:00.324 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-25T15:16:00.335 DEBUG:teuthology.orchestra.console.smithi137:power off output: Chassis Power Control: Down/Off 2024-04-25T15:16:00.335 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:00.353 INFO:teuthology.orchestra.console.smithi138:Power off 2024-04-25T15:16:00.354 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T15:16:00.364 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-04-25T15:16:00.380 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-04-25T15:16:00.380 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:00.405 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-25T15:16:04.325 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:04.366 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:04.406 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:04.439 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-04-25T15:16:04.482 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is on 2024-04-25T15:16:04.519 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-25T15:16:08.444 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:08.482 DEBUG:teuthology.orchestra.console.smithi137:pexpect command: ipmitool -H smithi137.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:08.520 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T15:16:08.558 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-04-25T15:16:08.594 DEBUG:teuthology.orchestra.console.smithi137:check power output: Chassis Power is off 2024-04-25T15:16:08.631 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-25T15:16:08.658 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-04-25T15:16:08.695 INFO:teuthology.orchestra.console.smithi137:Power off completed 2024-04-25T15:16:08.732 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-04-25T15:16:08.841 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 2045.8968069553375 failure_reason: 'Command failed on smithi094 with status 22: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get run_dir''' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b8a21f783bb848eb93ca47e55fbcaf13 status: fail success: false 2024-04-25T15:16:08.841 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T15:16:08.935 INFO:teuthology.run:FAIL